Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,297 records · Page 72
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Aspen Open Jets: unlocking LHC data for foundation models in particle physics

Foundation models are deep learning models pre-trained on large amounts of data which are capable of generalizing to multiple datasets and/or downstream tasks. This work demonstrates how data collected by the CMS experiment at the Large Hadron Collider can be useful in pre-training foundation models for HEP. Specifically, we introduce the AspenOpenJets (AOJs) dataset, consisting of approximately 178 M high p T jets derived from CMS 2016 Open Data. We show how pre-training the OmniJet-α foundation model on AOJs improves performance on generative tasks with significant domain shift: generating boosted top and QCD jets from the simulated JetClass dataset. In addition to demonstrating the power of pre-training of a jet-based foundation model on actual proton–proton collision data, we provide the ML-ready derived AOJs dataset for further public use.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

SIDDA: SInkhorn Dynamic Domain Adaptation for image classification with equivariant neural networks

Modern neural networks (NNs) often do not generalize well in the presence of a ‘covariate shift’; that is, in situations where the training and test data distributions differ, but the conditional distribution of classification labels given the data remains unchanged. In such cases, NN generalization can be reduced to a problem of learning more robust, domain-invariant features. Domain adaptation (DA) methods include a broad range of techniques aimed at achieving this; however, these methods have struggled with the need for extensive hyperparameter tuning, which then incurs significant computational costs. In this work, we introduce SInkhorn Dynamic Domain Adaptation (SIDDA), an out-of-the-box DA training algorithm built upon the Sinkhorn divergence, that can achieve effective domain alignment with minimal hyperparameter tuning and computational overhead. We demonstrate the efficacy of our method on multiple simulated and real datasets of varying complexity, including simple shapes, handwritten digits, real astronomical observations, and remote sensing data. These datasets exhibit covariate shifts due to noise, blurring, differences between telescopes, and variations in imaging wavelengths. SIDDA is compatible with a variety of NN architectures, and it works particularly well in improving classification accuracy and model calibration when paired with symmetry-aware equivariant NNs (ENNs). We find that SIDDA consistently enhances the generalization capabilities of NNs, achieving up to a ${\approx}40\%$ improvement in classification accuracy on unlabeled target data, while also providing a more modest performance gain of $\lesssim 1\%$ on labeled source data. We also study the efficacy of DA on ENNs with respect to the varying group orders of the dihedral group DN, and find that the model performance improves as the degree of equivariance increases. Finally, if SIDDA achieves proper domain alignment, it also enhances model calibration on both source and target data, with the most significant gains in the unlabeled target domain—achieving over an order of magnitude improvement in the expected calibration error and Brier score. SIDDA’s versatility across various NN models and datasets, combined with its automated approach to domain alignment, has the potential to significantly advance multi-dataset studies by enabling the development of highly generalizable models.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

D–MOPH–25: diverse MOF–molecule pairs for Henry’s constants prediction

Computational methods like grand-canonical Monte Carlo simulations and machine learning (ML) have accelerated metal–organic frameworks (MOF) exploration but are typically limited to a narrow range of adsorbates due to data availability and force field constraints. In this study, we introduce a dataset of diverse MOF–molecule pairs for Henry’s constant prediction, D–MOPH–25, which systematically explores a diverse chemical space by combining 113 molecular adsorbates with over 5000 MOF structures through an active learning process. D–MOPH–25 constitutes the most diverse adsorbate dataset used in any ML study of molecular adsorption in MOFs to date. Our workflow builds a benchmark for predicting Henry’s constants at 300 K, leveraging conformal prediction for uncertainty quantification. Assessment through Shannon entropy and uniform manifold approximation and projection confirms the comprehensiveness of D–MOPH–25 while highlighting the importance of robust classification to filter out unphysical data points in regression tasks. Although future enhancements in model architecture and sampling criteria could improve predictive performance, our dataset already spans the target space using only 2.31% of total possibilities. This comprehensive dataset facilitates assessment of model generalizability across adsorbate species and can establish a foundation for high-throughput MOF screening and ML-driven separation processes.

active learning↗
DOE OSTI2023

PYK-SubstitutionOME: an integrated database containing allosteric coupling, ligand affinity and mutational, structural, pathological, bioinformatic and computational information about pyruvate kinase isozymes

Interpreting changes in patient genomes, understanding how viruses evolve and engineering novel protein function all depend on accurately predicting the functional outcomes that arise from amino acid substitutions. To that end, the development of first-generation prediction algorithms was guided by historic experimental datasets. However, these datasets were heavily biased toward substitutions at positions that have not changed much throughout evolution (i.e. conserved). Although newer datasets include substitutions at positions that span a range of evolutionary conservation scores, these data are largely derived from assays that agglomerate multiple aspects of function. To facilitate predictions from the foundational chemical properties of proteins, large substitution databases with biochemical characterizations of function are needed. We report here a database derived from mutational, biochemical, bioinformatic, structural, pathological and computational studies of a highly studied protein family—pyruvate kinase (PYK). A centerpiece of this database is the biochemical characterization—including quantitative evaluation of allosteric regulation—of the changes that accompany substitutions at positions that sample the full conservation range observed in the PYK family. We have used these data to facilitate critical advances in the foundational studies of allosteric regulation and protein evolution and as rigorous benchmarks for testing protein predictions. We trust that the collected dataset will be useful for the broader scientific community in the further development of prediction algorithms.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Identifying strengths and weaknesses of methods for computational network inference from single-cell RNA-seq data

Single-cell RNA-sequencing (scRNA-seq) offers unparalleled insight into the transcriptional programs of different cellular states by measuring the transcriptome of thousands of individual cells. An emerging problem in the analysis of scRNA-seq is the inference of transcriptional gene regulatory networks and a number of methods with different learning frameworks have been developed to address this problem. Here, we present an expanded benchmarking study of eleven recent network inference methods on seven published scRNA-seq datasets in human, mouse, and yeast considering different types of gold standard networks and evaluation metrics. We evaluate methods based on their computing requirements as well as on their ability to recover the network structure. We find that, while most methods have a modest recovery of experimentally derived interactions based on global metrics such as Area Under the Precision Recall curve, methods are able to capture targets of regulators that are relevant to the system under study. Among the top performing methods that use only expression were SCENIC, PIDC, MERLIN or Correlation. Addition of prior biological knowledge and the estimation of transcription factor activities resulted in the best overall performance with the Inferelator and MERLIN methods that use prior knowledge outperforming methods that use expression alone. We found that imputation for network inference did not improve network inference accuracy and could be detrimental. Comparisons of inferred networks for comparable bulk conditions showed that the networks inferred from scRNA-seq datasets are often better or at par with the networks inferred from bulk datasets. Our analysis should be beneficial in selecting methods for network inference. At the same time, this highlights the need for improved methods and better gold standards for regulatory network inference from scRNAseq datasets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Meta-analysis identifies pleiotropic loci controlling phenotypic trade-offs in sorghum

Abstract Community association populations are composed of phenotypically and genetically diverse accessions. Once these populations are genotyped, the resulting marker data can be reused by different groups investigating the genetic basis of different traits. Because the same genotypes are observed and scored for a wide range of traits in different environments, these populations represent a unique resource to investigate pleiotropy. Here, we assembled a set of 234 separate trait datasets for the Sorghum Association Panel, a group of 406 sorghum genotypes widely employed by the sorghum genetics community. Comparison of genome-wide association studies (GWAS) conducted with two independently generated marker sets for this population demonstrate that existing genetic marker sets do not saturate the genome and likely capture only 35–43% of potentially detectable loci controlling variation for traits scored in this population. While limited evidence for pleiotropy was apparent in cross-GWAS comparisons, a multivariate adaptive shrinkage approach recovered both known pleiotropic effects of existing loci and new pleiotropic effects, particularly significant impacts of known dwarfing genes on root architecture. In addition, we identified new loci with pleiotropic effects consistent with known trade-offs in sorghum development. These results demonstrate the potential for mining existing trait datasets from widely used community association populations to enable new discoveries from existing trait datasets as new, denser genetic marker datasets are generated for existing community association populations.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

CORAL: A framework for rigorous self-validated data modeling and integrative, reproducible data analysis

Abstract Background Many organizations face challenges in managing and analyzing data, especially when relevant datasets arise from multiple sources and methods. Analyzing heterogeneous datasets and additional derived data requires rigorous tracking of their interrelationships and provenance. This task has long been a Grand Challenge of data science and has more recently been formalized in the FAIR principles: that all data objects be Findable, Accessible, Interoperable, and Reusable, both for machines and for people. Adherence to these principles is necessary for proper stewardship of information, for testing regulatory compliance, for measuring the efficiency of processes, and for facilitating reuse of data-analytical frameworks. Findings We present the Contextual Ontology-based Repository Analysis Library (CORAL), a platform that greatly facilitates adherence to all 4 of the FAIR principles, including the especially difficult challenge of making heterogeneous datasets Interoperable and Reusable across all parts of a large, long-lasting organization. To achieve this, CORAL's data model requires that data generators extensively document the context for all data, and our tools maintain that context throughout the entire analysis pipeline. CORAL also features a web interface for data generators to upload and explore data, as well as a Jupyter notebook interface for data analysts, both backed by a common API. Conclusions CORAL enables organizations to build FAIR data types on the fly as they are needed, avoiding the expense of bespoke data modeling. CORAL provides a uniquely powerful platform to enable integrative cross-dataset analyses, generating deeper insights than are possible using traditional analysis tools.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

End-to-End Automated Segmentation Framework for Four-Dimensional Scanning Transmission Electron Microscopy Data

Four-dimensional scanning transmission electron microscopy (4D-STEM) is powerful for rapidly characterizing arrays of nanoparticles produced via high-throughput synthesis. However, such 4D-STEM datasets typically contain thousands of nanoparticles, each characterized by thousands of diffraction patterns spatially distributed across the nanoparticle, necessitating efficient and comprehensive analysis. We propose an end-to-end segmentation framework to automatically segment each nanoparticle into regions with distinct composition/orientation of crystal grains, using only the 4D-STEM data. Bragg disk information is extracted in a physics-informed manner from the diffraction patterns at each spatial location and combined with the real space coordinates to form feature vectors. These feature vectors are then used as inputs to a Gaussian mixture model (GMM) to segment the nanoparticle into distinct regions. We also develop two visualization tools based on the GMM outputs to infer the interface transition and the degree of superposition. Our framework comprehensively integrates machine learning tools and physics knowledge, and provides a basis for substantially compressing enormous 4D-STEM datasets, e.g., by replacing the full 4D-STEM dataset for each nanoparticle with only a single set of Bragg disk features for each distinct crystal grain identified in the nanoparticle. In this article, we demonstrate the power of our framework by presenting results for real, complex datasets.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Neutrino-nucleus CC 0 π cross-section tuning in GENIE v3

This article summarizes the state of the art of v μ and $\bar{v}_μ$ CC0π cross-section measurements on carbon and argon and discusses the relevant nuclear models, parametrizations and uncertainties in GENIE v3. The CC0π event topology is common in experiments at a few-GeV energy range. Although its main contribution comes from quasi-elastic interactions, this topology is still not well understood. The GENIE global analysis framework is exploited to analyze CC0π datasets from MiniBooNE, T2K and MINERvA. A partial tune for each experiment is performed, providing a common base for the discussion of tensions between datasets. The results offer an improved description of nuclear CC0π datasets as well as data-driven uncertainties for each experiment. This work is a step towards a GENIE global tune that improves our understanding of neutrino interactions on nuclei. It follows from earlier GENIE work on the analysis of neutrino scattering datasets on hydrogen and deuterium.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Quantifying scalar field dynamics with DESI 2024 Y1 BAO measurements

Quintessence scalar fields are a natural candidate for evolving dark energy. Unlike the phenomenological w 0 ⁢w a parameterization of the dark energy equation of state, they cannot accommodate the phantom regime of dark energy w⁡(z) <–1, or crossings into the phantom regime. Recent baryon acoustic oscillation (BAO) measurements by the Dark Energy Spectroscopic Instrument (DESI) indicate a preference for evolving dark energy over a cosmological constant, ranging from 2.6⁢σ –3.9⁢σ when fitting to w 0⁢ w a , and combining the DESI BAO measurements with other cosmological probes. In this work, we directly fit three simple scalar field models to the DESI BAO data, combined with cosmic microwave background anisotropy measurements and supernova datasets. We find the best fit model to include a 2–4% kinetic scalar field energy Ω scf,k , for a canonical scalar field with a quadratic or linear potential. However, only the DESY-Y5 supernova dataset combination shows a preference for quintessence over Λ cold dark matter (CDM) at the 95% confidence level. Fitting to the supernova datasets Pantheon, Pantheon+, DES-Y5, and Union3, we show that the mild tension (n σ < 3.4) under Λ ⁢CDM emerges from a BAO preference for smaller values of fractional mass-energy density Ω m < 0.29, while all supernova datasets, except for Pantheon, prefer larger values, Ω m > 0.3. The tension under Λ ⁢CDM remains noticeable (n σ < 2.8), when replacing two of the DESI BAO redshift bins with effective redshifts z eff = 0.51, and z eff = 0.706 with comparable BOSS DR 12 BAO measurements at z eff = 0.51, and z eff = 0.61. Furthermore, canonical scalar fields as dark energy are successful in mitigating that tension.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

NeuralCubes: Deep Representations for Visual Data Exploration

Visual exploration of large multi-dimensional datasets has seen tremendous progress in recent years, allowing users to express rich data queries that produce informative visual summaries, all in real time. Techniques based on data cubes are some of the most promising approaches. However, these techniques usually require a large memory footprint for large datasets. To tackle this problem, we present NeuralCubes: neural networks that predict results for aggregate queries, similar to data cubes. NeuralCubes learns a function that takes as input a given query, for instance, a geographic region and temporal interval, and outputs the result of the query. The learned function serves as a real-time, low-memory approximator for aggregation queries. Our models are small enough to be sent to the client side (e.g. the web browser for a web-based application) for evaluation, enabling data exploration of large datasets without database/network connection. Here, we demonstrate the effectiveness of NeuralCubes through extensive experiments on a variety of datasets and discuss how NeuralCubes opens up opportunities for new types of visualization and interaction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Tuning the Interpolation Basis in a Multigrid Decomposition for Local Error Control

In the compression of scientific data, error-controlled compressors enable to considerably decrease the size of the dataset while maintaining adequate levels of accuracy. In this paper, we note that multi-level refactoring scheme such as MGARD i) rely on an approximation of the data based on the interpolation of coefficients, ii) estimate the resulting error with global metrics on the dataset. To improve on these two aspects, we propose a method that aims to divide the original dataset into blocks based on their smoothness and refactors each block separately with the most relevant interpolation order. We show the relevance of such a method on tailored datasets and the benefits and challenges when applying it to large scientific data.

Vidal, Nicolas [ORNL]↗
DOE OSTI2024

DaYu: Optimizing Distributed Scientific Workflows by Decoding Dataflow Semantics and Dynamics

The combination of ever-growing scientific datasets and distributed workflow complexity creates I/O performance bottlenecks due to data volume, velocity, and variety. Although the increasing use of descriptive data formats (e.g., HDF5, netCDF) helps organize these datasets, it also creates obscure bottlenecks due to the need to translate high level operations into file addresses and then into low-level I/O operations. To address this challenge, we introduce DaYu, a method and toolset for analyzing (a) semantic relationships between logical datasets and file addresses, (b) how dataset operations translate into I/O, and (c) the combination across entire workflows. DaYu's analysis and visualization enables identification of critical bottlenecks and reasoning about remediation. We describe our methodology and propose optimization guidelines. Evaluation on scientific workflows demonstrates up to 3.7x performance improvements in I/O time for obscure bottlenecks. The time and storage overhead for DaYu's time-ordered data is typically under 0.2% of runtime and 0.25% of data volume, respectively.

Tang, Meng↗
DOE OSTI2026

VoroClust: Scalable Clustering for Remote Sensing

Although supervised machine learning provides a powerful framework for image classification and segmentation, it requires comprehensive consistent datasets, which are not available for many remote-sensing applications. Remote-sensing datasets are expensive to collect, and each is acquired under different environmental conditions or with significant variations in system operating parameters. Unsupervised clustering algorithms analyze the structure of each dataset independently, rather than drawing on similarities with existing “training” examples, and are thus well suited for practical remote-sensing applications. We introduce VoroClust, a fast density-based unsupervised clustering algorithm applicable to high-resolution and high-dimensional data. VoroClust runs as fast as distance-based clustering methods, while capturing complex regional geometries at least as well as current-density-based methods. It uses a data-centered sphere cover to reduce computational demands, while still capturing data topology. It then propagates clusters outward from local peaks in density. We show that VoroClust provides fast state-of-the-art clustering for both high-resolution polarimetric synthetic aperture radar and high-dimensional hyperspectral imaging datasets.

42 ENGINEERING↗
DOE OSTI2022

A Comparison of Machine Learning Methods for Frequency Nadir Estimation in Power Systems

An increasing penetration level of inverter-based renewable energy resources changes the inertia of power systems, posing challenges for maintaining the desired system frequency stability. An accurate frequency nadir estimation is crucial for power system operators to prepare preventive actions against large frequency excursions. In this paper, five machine learning methods - linear regression, gradient boosting, support vector regression, an artificial neural network, and XGBoost - are applied to two different datasets, i.e., 1) the unit generation dataset and 2) the system total inertia and headroom dataset, for the prediction of the frequency nadir. The training and testing datasets are generated through extensive generation scheduling simulations using Multi-timescale Integrated Dynamic and Scheduling (MI-DAS) toolbox on the Western Electricity Coordinating Council 240-bus system with high renewable penetration levels. Numerical results show that all five machine learning methods perform well in predicting the nadir frequency of the system. Among them, the gradient boosting and the XGBoost are clear winners yielding the best prediction accuracy in terms of four evaluation metrics.

data driven↗
DOE OSTI2021

Controllable Reset Behavior in Domain Wall–Magnetic Tunnel Junction Artificial Neurons for Task-Adaptable Computation

Neuromorphic computing with spintronic devices has been of interest due to the limitations of CMOS-driven von Neumann computing. Domain wall–magnetic tunnel junction (DW-MTJ) devices have been shown to be able to intrinsically capture biological neuron behavior. Edgy-relaxed behavior, where a frequently firing neuron experiences a lower action potential threshold, may provide additional artificial neuronal functionality when executing repeated tasks. In this letter, we demonstrate that this behavior can be implemented in DW-MTJ artificial neurons via three alternative mechanisms: shape anisotropy, magnetic field, and current-driven soft reset. Using micromagnetics and analytical device modeling to classify the Optdigits handwritten digit dataset, we show that edgy-relaxed behavior improves both classification accuracy and classification rate for ordered datasets while sacrificing little to no accuracy for a randomized dataset. This letter establishes methods by which artificial spintronic neurons can be flexibly adapted to datasets.

42 ENGINEERING↗
DOE OSTI2024

MetaPoL: Immersive VR based Indoor Patterns of Life (PoL) and Anomalies Data Generation for Insider Threat Modeling in Nuclear Security

Insider threats are perhaps the most serious challenges that nuclear and radiological security systems face. Insiders pose such a great threat due to their access, authority, and knowledge, granting them opportunities to bypass dedicated nuclear and radiological security elements. For example, in one of the latest major insider threat incidents to nuclear security, the Doel-4 nuclear powerplant in Belgium suffered a shutdown, the threat of nuclear materials diversion, and long-term loss of tens of millions of dollars. Seven years of investigation concluded that it was an inside job and attempted sabotage. In this regard, there is an immediate need for R&D and technology integration in the domain of modeling indoor Patterns-of-Life (PoL) and anomaly detection. This can be achieved by using datasets of facility users’ mobility and activity, which can support the design of algorithms for insider threat modeling and detection. However, due to classification, privacy, sensitivity, and safety protocols, such datasets from real physical nuclear reactor facilities are not only hard to share, but also not always feasible to deploy and collect. Aiming to find an alternate solution, our proposed demonstration work - MetaPoL, is the first-ever (for the application space) immersive VR (virtual reality) environment of a real-world secure facility and allows users to move-and-stay through the designed indoor physical layout and also encounter NPCs (non-player characters) that emulate other facility users. In the MetaPoL an interactive user performs realistic spatio-temporal movement, dwelling and activities using a Meta Quest Pro VR headset, and that generates high-frequency (in time) high-resolution (in space) indoor spatial-temporal datasets that are valuable for PoL modeling and anomaly detection research specifically for insider threat modeling and detection mission. Such generated realistic, rich in context, and mission specific datasets can boost AI/Machine Learning based research for modeling and detecting insider threats in nuclear security and nonproliferation.

Gunaratne, Chathika↗
DOE OSTI2026

“One Table to Rule Them All”: How a Single Table can Enable Extensive Insights, Analytics and Assessment on Human Mobility Data

While much research has been conducted in Human Mobility Science, most studies on the analytics/insights part generally focus on one of the following: processing and analytics on human stop-trip behavior, design of individual mobility metrics (often in silos), calculation and characterization of only a handful (typically 5-6) of human mobility metrics on geospatial-temporal human mobility data of interest. Although human mobility research offers a vast and diverse array of available metrics, most individual studies typically compute only a small subset of five or six metrics at a time when analyzing trajectory datasets of human mobility across different areas of interest. This paper is motivated by the critical need to repeatedly compute an extensive array of human mobility metrics across several trajectory datasets and perform individual metric-level benchmarking to establish a new, standardized Test and Evaluation (T&E) suite for the field of Human Mobility Science. We first present our findings on the minimal yet sufficient pre-processing required to reliably and efficiently compute a wide range of human mobility metrics. The key findings are specifically related to the proposed Composite Stop Locations table, which serves as a core pre-processing data layer. Subsequently, we present a case study demonstrating how the Composite Stop Locations table facilitates computation of at least 14 distinct human mobility metrics (unlike 5-6 different set of metrics used for studies in the literature) using the popular and open-source OpenPFLOW dataset. Finally, we have also presented an example of our benchmarking methodology to evaluate the quality and performance of the trajectory dataset of interest, assessed across multiple human mobility metrics.

De, Debraj [ORNL] (ORCID:0000000233630020)↗