Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

36 MATERIALS SCIENCE↗

Good practices for documenting AI-based studies on energy and buildings

Artificial intelligence has transformed building science research over the past decade, with applications spanning energy modeling, energy prediction, HVAC optimization and controls, fault detection, and occupancy modeling. However, many studies lack adequate documentation of datasets, algorithms, training procedures, and validation methods. Building science research faces additional challenges including inconsistent evaluation metrics, limited generalizability across building types, climates, and significant gaps between experimental studies and deployed systems. This communication provides practical guidance for good practices in documenting and publishing AI-based research following established standards from the computer science and machine learning communities. By adopting frameworks such as Datasheets for Datasets, Model Cards, and standardized reproducibility checklists, researchers can ensure their work meets the rigorous documentation standards necessary for reproducible, comparable, and impactful building science research.

Hong, Tianzhen [Lawrence Berkeley National Laborat↗

Missing microbial eukaryotes and misleading meta-omic conclusions

Meta-omics is commonly used for large-scale analyses of microbial eukaryotes, including species or taxonomic group distribution mapping, gene catalog construction, and inference on the functional roles and activities of microbial eukaryotes in situ. Here, we explore the potential pitfalls of common approaches to taxonomic annotation of protistan meta-omic datasets. We re-analyze three environmental datasets at three levels of taxonomic hierarchy in order to illustrate the crucial importance of database completeness and curation in enabling accurate environmental interpretation. We show that taxonomic membership of sequence clusters estimates community composition more accurately than returning exact sequence labels, and overlap between clusters can address database shortcomings. Clustering approaches can be applied to diverse environments while continuing to exploit the wealth of annotation data collated in databases, and selecting and evaluating these databases is a critical part of correctly annotating protistan taxonomy in environmental datasets. We argue that ongoing curation of genetic resources is crucial in accurately annotating protists in in situ meta-omic datasets. Moreover, we propose that precise taxonomic annotation of meta-omic data is a clustering problem rather than a feasible alignment problem.

59 BASIC BIOLOGICAL SCIENCES↗

Identification and Clinical Evaluation of Potential Biomarkers for Breast Cancer Resistance Protein ( BCRP / ABCG2 )

Clinical inhibition and genetic variation of the Breast Cancer Resistance Protein (BCRP/ABCG2) efflux transporter can significantly influence drug exposure, highlighting the need for reliable BCRP functional biomarkers. This study aimed to identify and evaluate biomarkers predictive of BCRP function in humans. A comprehensive analysis of metabolomic genome‐wide association studies (mGWAS) was conducted to discover potential BCRP biomarkers, followed by evaluation inin vitrotransporter assays and a clinical drug–drug interaction (DDI) study. Across multiple mGWAS datasets, plasma concentrations of three herbicide derivatives—4‐hydroxychlorothalonil (4HC), 3‐bromo‐5‐chloro‐2,6‐dihydroxybenzoic acid (BCDBA), and 3,5‐dichloro‐2,6‐dihydroxybenzoic acid (DCDBA)—were significantly elevated (P < 5E‐8) in individuals carrying reduced functionABCG2polymorphisms. These compounds were confirmed as novel BCRP substrates via transporter uptake assays and selected for clinical evaluation alongside riboflavin, a known BCRP substrate and potential BCRP biomarker. In a DDI study with 11 healthy subjects, eltrombopag, a BCRP inhibitor, increased rosuvastatin concentrations by approximately twofold (P = 0.002). No significant changes in the plasma concentrations of organic anion transporting polypeptide 1B (OATP1B) biomarkers (CP‐I and CP‐III) or potential BCRP biomarkers (4HC, BCDBA, DCDBA, or riboflavin) were observed. Notably, two subjects were heterozygous carriers for theABCG2p.Q141K variant and exhibited significantly higher baseline concentrations of 4HC (P = 0.004) and BCDBA (P = 0.0003), consistent with reduced BCRP function. These findings suggest that 4HC and BCDBA are promising biomarkers for baseline BCRP function in specific populations, such as those harboring reduced function genetic polymorphisms, but do not appear suitable for detecting acute BCRP inhibition.

Pharmacology & Pharmacy↗

Evaluation of a high-resolution regional climate simulation for surface and hub-height wind climatology over North America

Assessing the availability of key wind resources requires augmenting observations to support the implementation of wind energy infrastructure. However, observations are limited, necessitating the development of high-resolution, long-term gridded datasets. This study presents a robust, dynamically downscaled climatological dataset, offering 20 years of hourly wind data at a 4 km spatial resolution across North America, and evaluates its performance against observations, including meteorological towers and automated surface-observing system (ASOS) stations, as well as coarse-resolution reanalysis data (the European Centre for Medium-Range Weather Forecasts (ECMWF) reanalysis version 5 (ERA5)). Results demonstrate that the downscaled high-resolution wind data outperform ERA5 in regions of complex terrain and coastal areas, with improved overlap coefficients for wind data distributions and reduced root mean square errors (RMSEs) for hub-height and near-surface diurnal wind patterns. The downscaled simulation also captures the synoptic drivers of seasonal wind direction patterns reasonably well, indicated by high wind rose similarity indices. This study also provides an analysis of interannual variability, utilizing the dataset's full 20-year period, and model uncertainty, generated by varying model initial conditions and physics parameterizations across 1-year ensemble members, which are key considerations for wind resource assessment in wind farm development.

17 WIND ENERGY↗

CoCoMET v1.0: a unified open-source toolkit for atmospheric object tracking and analysis

Advances in performance and analysis capabilities have accelerated the development of object tracking algorithms for atmospheric research. This has resulted in a growing number of studies using Lagrangian tracking techniques to analyze the evolution of atmospheric phenomena and the underlying processes. However, the increasing complexity and variety of tracking algorithms present a steep learning curve for new users and make it difficult for existing users to compare algorithm performance. We introduce CoCoMET (Community Cloud Model Evaluation Toolkit), an open-source toolkit that addresses these issues. CoCoMET simplifies the process of running multiple tracking algorithms simultaneously and analyzing objects in both model and observational datasets by specifying parameters in a single configuration file. It standardizes input data from different sources into a consistent format and unifies the tracking output across algorithms. CoCoMET enhances the functionality of existing tracking methods by calculating additional properties such as cell growth and dissipation rates, perimeter, surface area, convexity, and irregularity. In addition, CoCoMET includes a novel method for identifying mergers and splits in 2D and 3D tracks and supports the integration of Eulerian/stationary datasets external to the tracking data for process studies. Its potential utility is demonstrated through examples of model intercomparison, model evaluation against observations, and comparisons between tracking algorithms. Designed for open-source environments, CoCoMET will continue to expand with future releases, incorporating more input data types and tracking algorithms.

54 ENVIRONMENTAL SCIENCES↗

Low-Temperature Geothermal Geospatial Datasets: An Example from Alaska

This project is a component of a broader effort focused on geothermal heating and cooling (GHC) with the aim of illustrating the numerous benefits of incorporating GHC and geothermal heat exchange (GHX) into community energy planning and national decarbonization strategies. To better assist private sector investment, it is currently necessary to define and assess the potential of low-temperature geothermal resources. For shallow GHC/GHX fields, there is no formal compilation of subsurface characteristics shared among industry practitioners that can improve system design and operations. Alaska is specifically noted in this work, because heretofore, it has not received a similar focus in geothermal potential evaluations as the contiguous United States. The methodology consists of leveraging relevant data to generate a baseline geospatial dataset of low-temperature resources (less than 150 degrees C) to compare and analyze information accessible to anyone trying to understand the potential of GHC/GHX and small-scale low-temperature geothermal power in Alaska (e.g., energy modelers, communities, planners, and policymakers). Importantly, this project identifies data related to (1) the evaluation of GHC/GHX in the shallow subsurface, and (2) the evaluation of low-temperature geothermal resource availability. Additionally, data is being compiled to assess repurposing of oil and gas wells to contribute co-produced fluids toward the geothermal direct use and heating and cooling resource potential. In this work we identified new data from three different datasets of isolated geothermal systems in Alaska and bottom-hole temperature data from oil and gas wells that can be leveraged for evaluation of low-temperature geothermal resource potential. The goal of this project is to facilitate future deployment of GHC/GHX analysis and community-led programs and update the low-temperature geothermal resources assessment of Alaska. A better understanding of shallow potential for GHX will improve design and operations of highly efficient GHC systems. The deployment and impact that can be achieved for low-temperature geothermal resources will contribute to decarbonization goals and facilitate widespread electrification by shaving and shifting grid loads.

15 GEOTHERMAL ENERGY↗

Including 238 U(n,f)/ 235 U(n,f) and 239 Pu(n,f)/ 235 U(n,f) NIFFTE fission TPC Cross-sections into the Neutron Data Standards Database

The primary purpose of this report is to document how the 238 U/ 235 U and 239 Pu/ 235 U neutron induced fission cross-section ratios, 238 U(n,f)/ 235 U(n,f) and 239 Pu(n,f)/ 235 U(n,f), respectively, measured by the NIFFTE fission Time Projection Chamber (fissionTPC) were included in the most recent database (termed GMA) underlying Neutron Data Standards (NDS) evaluations. This report shows and discusses NDS input files, and underlying assumptions regarding the uncertainty estimate and necessary for including these data. This uncertainty estimate and the resulting files were based on information provided by fissionTPC experimentalists, R.J.Casperson, N.S. Bowden, L. Snyder and K.T. Schmitt for the 238 U ratio, and by L. Snyder for the 239 Pu ratio. The fissionTPC data were included twice, by D. Neudecker and V. Pronyaev, to counter-check results and exclude possible mistakes in their inclusion. It is shown in both evaluations that including fissionTPC 239 Pu(n,f)/ 235 U(n,f) data points to a lower evaluated 239 Pu(n,f) cross section above 10 MeV than the currently released NDS data. This raises the question whether a part of a previous dataset by Tovesson et al., that was previously rejected above 13 MeV for having low values, should be included in the NDS evaluation after all. The evaluated 238 U(n,f) cross section only changes significantly close to the threshold. The impact on the 235 U(n,f) cross section is minimal. fissionTPC data reduce evaluated uncertainties on both observables by 0–12% of the GMA evaluated uncertainties. However, the currently released NDS data contain in addition to these GMA evaluated uncertainties “Unrecognized Sources of Uncertainties” (USU) of 1.2%. It needs to be further discussed within the NDS project, whether the new fissionTPC data should also reduce USU.

238U(n,f)/235U(n,f)↗

Finite Element Analysis and Machine Learning Guided Design of Carbon Fiber Organosheet-Based Battery Enclosures for Crashworthiness

Carbon fiber composite can be a potential candidate for replacing metal-based battery enclosures of current electric vehicles (E.V.s) owing to its better strength-to-weight ratio and corrosion resistance. However, the strength of carbon fiber-based structures depends on several parameters that should be carefully chosen. Here, in this work, we implemented high throughput finite element analysis (FEA) based thermoforming simulation to virtually manufacture the battery enclosure using different design and processing parameters. Subsequently, we performed virtual crash simulations to mimic a side pole crash to evaluate the crashworthiness of the battery enclosures. This high throughput crash simulation dataset was utilized to build predictive models to understand the crashworthiness of an unknown set. Our machine learning (ML) models showed excellent performance (R 2 > 0.97) in predicting the crashworthiness metrics, i.e., crush load efficiency, absorbed energy, intrusion, and maximum deceleration during a crash. We believe that this FEA-ML work framework will be helpful in down select process parameters for carbon fiber-based component design and can be transferrable to other manufacturing technologies.

36 MATERIALS SCIENCE↗

Programmable simulations of molecules and materials with reconfigurable quantum processors

Simulations of quantum chemistry and quantum materials are believed to be among the most important applications of quantum information processors. However, realizing practical quantum advantage for such problems is challenging because of the prohibitive computational cost of programming typical problems into quantum hardware. Here we introduce a simulation framework for strongly correlated quantum systems represented by model spin Hamiltonians that uses reconfigurable qubit architectures to simulate real-time dynamics in a programmable way. Our approach also introduces an algorithm for extracting chemically relevant spectral properties via classical co-processing of quantum measurement results. We develop a digital–analogue simulation toolbox for efficient Hamiltonian time evolution using digital Floquet engineering and hardware-optimized multi-qubit operations to accurately realize complex spin–spin interactions. As an example, we propose an implementation based on Rydberg atom arrays. In addition, we show how detailed spectral information can be extracted from the dynamics through snapshot measurements and single-ancilla control, enabling the evaluation of excitation energies and finite-temperature susceptibilities from a single dataset. To illustrate the approach, we show how to use the method to compute key properties of a polynuclear transition-metal catalyst and two-dimensional magnetic materials.

74 ATOMIC AND MOLECULAR PHYSICS↗

CaloScore v2: single-shot calorimeter shower simulation with diffusion models

Diffusion generative models are promising alternatives for fast surrogate models, producing high-fidelity physics simulations. However, the generation time often requires an expensive denoising process with hundreds of function evaluations, restricting the current applicability of these models in a realistic setting. In this work, we report updates on the CaloScorearchitecture, detailing the changes in the diffusion process, which produces higher quality samples, and the use of progressive distillation, resulting in a diffusion model capable of generating new samples with a single function evaluation. Here we demonstrate these improvements using the Calorimeter Simulation Challenge 2022 dataset.

47 OTHER INSTRUMENTATION↗

Applications of physics informed neural operators

Abstract We present a critical analysis of physics-informed neural operators (PINOs) to solve partial differential equations (PDEs) that are ubiquitous in the study and modeling of physics phenomena using carefully curated datasets. Further, we provide a benchmarking suite which can be used to evaluate PINOs in solving such problems. We first demonstrate that our methods reproduce the accuracy and performance of other neural operators published elsewhere in the literature to learn the 1D wave equation and the 1D Burgers equation. Thereafter, we apply our PINOs to learn new types of equations, including the 2D Burgers equation in the scalar, inviscid and vector types. Finally, we show that our approach is also applicable to learn the physics of the 2D linear and nonlinear shallow water equations, which involve three coupled PDEs. We release our artificial intelligence surrogates and scientific software to produce initial data and boundary conditions to study a broad range of physically motivated scenarios. We provide the source code , an interactive website to visualize the predictions of our PINOs, and a tutorial for their use at the Data and Learning Hub for Science .

97 MATHEMATICS AND COMPUTING↗

Fair Concurrent Training of Multiple Models in Federated Learning

Federated learning (FL) enables collaborative learning across multiple clients. In most FL work, all clients train a single learning task. However, the recent proliferation of FL applications may increasingly require multiple FL tasks to be trained simultaneously, sharing clients’ computing resources, which we call Multiple-Model Federated Learning (MMFL). Current MMFL algorithms use naïve average-based client-task allocation schemes that often lead to unfair performance when FL tasks have heterogeneous difficulty levels, as the more difficult tasks may need more client participation to train effectively. Furthermore, in the MMFL setting, we face a further challenge that some clients may prefer training specific tasks to others, and may not even be willing to train other tasks, e.g., due to high computational costs, which may exacerbate unfairness in training outcomes across tasks. We address both challenges by firstly designing FedFairMMFL, a difficulty-aware algorithm that dynamically allocates clients to tasks in each training round, based on the tasks’ current performance levels. We provide guarantees on the resulting task fairness and FedFairMMFL’s convergence rate. We then propose novel auction designs that incentivizes clients to train multiple tasks, so as to fairly distribute clients’ training efforts across the tasks, and extend our convergence guarantees to this setting. Here, we finally evaluate our algorithm with multiple sets of learning tasks on real world datasets, showing that our algorithm improves fairness by improving the final model accuracy and convergence speed of the worst performing tasks, while maintaining the average accuracy across tasks.

Federated learning↗

Continental Scale Hydrostratigraphy: Comparing Geologically Informed Data Products to Analytical Solutions

Abstract This study synthesizes two different methods for estimating hydraulic conductivity (K) at large scales. We derive analytical approaches that estimate K and apply them to the contiguous United States. We then compare these analytical approaches to three‐dimensional, national gridded K data products and three transmissivity (T) data products developed from publicly available sources. We evaluate these data products using multiple approaches: comparing their statistics qualitatively and quantitatively and with hydrologic model simulations. Some of these datasets were used as inputs for an integrated hydrologic model of the Upper Colorado River Basin and the comparison of the results with observations was used to further evaluate the K data products. Simulated average daily streamflow was compared to daily flow data from 10 USGS stream gages in the domain, and annually averaged simulated groundwater depths are compared to observations from nearly 2000 monitoring wells. We find streamflow predictions from analytically informed simulations to be similar in relative bias and Spearman's rho to the geologically informed simulations. R ‐squared values for groundwater depth predictions are close between the best performing analytically and geologically informed simulations at 0.68 and 0.70 respectively, with RMSE values under 10 m. We also show that the analytical approach derived by this study produces estimates of K that are similar in spatial distribution, standard deviation, mean value, and modeling performance to geologically‐informed estimates. The results of this work are used to inform a follow‐on study that tests additional data‐driven approaches in multiple basins within the contiguous United States.

54 ENVIRONMENTAL SCIENCES↗

Robustness of topological persistence in knowledge distillation for wearable sensor data

Topological data analysis (TDA) has shown great success in various applications involving wearable sensor data. However, there are difficulties in leveraging topological features in machine learning and wearable sensors because of the large time consumption and computational resources required to extract the features. To address this problem, knowledge distillation (KD) is utilized to generate a small model and accommodate topological features with persistence image (PI) representations from the raw time series data. Deploying topological knowledge in KD enables the student to achieve better performance compared to the one trained solely on raw time series data. However, it is not yet known if there are coherent characteristics for topological features in PI, which can aid in improving the performance during KD. In this paper, we investigate the suitability and challenges of utilizing topological features in KD for wearable sensor data, thereby contributing to the advancement of the field. Our study explores the impact of transferred topological features by comparing the Teacher-to-Student framework with Multiple Teachers-to-Student where teachers utilize both time series data and persistence images obtained by TDA as inputs. Additionally, we conduct a rigorous examination of topological knowledge effects by testing under various corruptions, knowledge types, and learning strategies in the context of human activity recognition tasks. Our analysis of topological features in KD presents the optimal strategy for incorporating these features. This study includes datasets of varying scales, window lengths, and activity classes, providing a comprehensive evaluation. Our results demonstrate that leveraging topological features in KD to enhance performance across databases.

97 MATHEMATICS AND COMPUTING↗

TransPlatformer

We propose TransPlatformer for translating toxicogenomics from one platform to another. Transcriptomic profiling has evolved through multiple generations of technology, from microarrays (e.g., Affymetrix, CodeLink) to more recent high-throughput sequencing and targeted panels such as S1500+. Microarrays, which dominated gene expression studies in the early 2000s, provided affordable and high-throughput transcript quantification but suffered from cross-hybridization issues and limited dynamic range . RNA-Seq, introduced in the late 2000s, revolutionized transcriptomics by enabling unbiased and comprehensive gene expression analysis, albeit at higher costs and computational demands . Despite advances, many studies rely on historical microarray data, necessitating the translation of legacy data into modern platforms to ensure continuity and comparability. This translation is complicated by factors such as platform-specific probe design, differences in transcript coverage, and batch effects . Existing methods for cross-platform mapping include statistical normalization, machine learning models, and biological anchoring approaches. The ability to translate transcriptomic data between platforms has broad implications, including enhanced meta-analyses, improved toxicological modeling, and better integration of historical datasets with contemporary research. TransPlatformer seeks to contribute to this effort by evaluating translation methodologies and proposing novel strategies to improve cross-platform gene expression harmonization. In this repository there are code examples for TransPlatformer implementation

Cong, Guojing↗

NLR VISSIM Wrapper [SWR-26-023]

Software that allows the execution of VISSIM simulations based on any airport access/egress mode share scenario. This is a standalone executable that takes the baseline VISSIM model provided by Seattle Tacoma International Airport (SEA) and a synthetic passenger dataset as inputs and generates travel time changes based on the congestion evaluation function built into the software.

Ge, Yanbo [National Laboratory of the Rockies (NLR↗

MLCommons Science Benchmarks

Benchmarks are a cornerstone of modern machine learning practice, providing standardized eval- uations that enable reproducibility, comparison, and scientific progress. Yet, as AI systems particularly deep learning models become increasingly dynamic, traditional static benchmarking approaches are losing their relevance. Models rapidly evolve in architecture, scale, and capability; datasets shift; and deployment contexts continuously change, creating a moving target for evaluation. Without adaptive benchmarking frame- works, both scientific assessment and real-world de- ployment risk becoming misaligned with actual system behavior. Drawing on our experience from MLCommons, educa- tional initiatives, and government programs such as the DOE s Million Parameter Consortium, we identify key barriers that hinder the broader adoption and utility of benchmarking in AI. These include substantial resource demands, limited access to specialized hardware, lack of expertise in benchmark design, and uncertainty among practitioners about how to relate benchmark results to their own application domains. Moreover, current benchmarks often emphasize peak performance on leadership-class hardware, offering limited guidance for more diverse, real-world deployment scenarios. We argue that benchmarking itself must become dy- namic in order to incorporate evolving models, updated data, and heterogeneous computational platforms while maintaining transparency, reproducibility, and inter- pretability. Democratizing this process requires not only technical innovation, but also systematic educational efforts spanning undergraduate to professional levels to develop sustained expertise in benchmark design and use. Finally, benchmarks should be framed and com- municated to support application-relevant comparisons, enabling both developers and users to make informed, context-sensitive decisions. Advancing dynamic and inclusive benchmarking practices will be essential to ensure that evaluation keeps pace with the evolving AI landscape and supports responsible, reproducible, and accessible AI deployment.

Hawks, Benjamin G. [Fermilab]↗