Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “massive datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Picasso: Memory-Efficient Graph Coloring Using Palettes With Applications in Quantum Computing

A coloring of a graph is an assignment of colors to vertices such that no two neighboring vertices have the same color. The need for memory-efficient coloring algorithms is motivated by their application in computing clique partitions of graphs arising in quantum computations where the objective is to map a large set of Pauli strings into a compact set of unitaries. We present Picasso, a randomized memory-efficient iterative parallel graph coloring algorithm with theoretical sublinear space guarantees under practical assumptions. The parameters of our algorithm provide a trade-off between coloring quality and resource consumption. To assist the user, we also propose a machine learning model to predict the coloring algorithm’s parameters considering these trade-offs. We provide a sequential and a parallel implementation of the proposed algorithm. We perform an experimental evaluation on a 64-core AMD CPU equipped with 512 GB of memory and an Nvidia A100 GPU with 40GB of memory. For a small dataset where existing coloring algorithms can be executed within the 512 GB memory budget, we show up to 68× memory savings. On massive datasets we demonstrate that GPU-accelerated Picasso can process inputs with 49.5× more Pauli strings (vertex set in our graph) and 2,478× more edges than state-of-the-art parallel approaches.

artificial intelligence, quantum computing↗
DOE OSTI2025

Frictionless knowledge injection for few-shot learning

Cutting-edge machine learning methods often require large volumes of curated training data, precluding their use in national security problems with rare events in massive datasets. We present a method for incorporating abstract knowledge into models tailored for sparse data. A subject matter expert defines salient concepts using data examples, which are encoded in the model’s embedding space. Models are then trained to respect these concepts. This method enables knowledge injection, yielding effective models with limited labeled data and the ability to assess model sensitivity for subject matter expertise across the nonproliferation mission space, as demonstrated with Raman spectra analysis.

Stomps, Jordan [ORNL] (ORCID:0000000178114479)↗
DOE OSTI2025

Scalable 3D reconstruction for X-ray single particle imaging with online machine learning

X-ray free-electron lasers offer unique capabilities for measuring the structure and dynamics of biomolecules, helping us understand the basic building blocks of life. Notably, high-repetition-rate free-electron lasers enable single particle imaging, where individual, weakly scattering biomolecules are imaged under near-physiological conditions with the opportunity to access fleeting states that cannot be captured in cryogenic or crystallized conditions. Existing X-ray single particle reconstruction algorithms, which estimate the particle orientation for each image independently, are slow and memory-intensive when handling the massive datasets generated by emerging free-electron lasers. Here, we introduce X-RAI (X-Ray single particle imaging with Amortized Inference), an online reconstruction framework that estimates the structure of 3D macromolecules from large X-ray single particle datasets. X-RAI consists of a convolutional encoder, which amortizes pose estimation over large datasets, as well as a physics-based decoder, which employs an implicit neural representation to enable high-quality 3D reconstruction in an end-to-end, self-supervised manner. We demonstrate that X-RAI achieves state-of-the-art performance for small-scale datasets in simulation and challenging experimental settings and demonstrate its unprecedented ability to process large datasets containing millions of diffraction images in an online fashion. These abilities signify a paradigm shift in X-ray single particle imaging towards real-time reconstruction.

Computer science↗
DOE OSTI2026

Foundation models for atomistic simulation of chemistry and materials

Conventional computational methods for modeling chemical and materials systems are limited by system size and timescale, forcing a trade-off between quantum-mechanical accuracy and the sampling needed for realistic observables. Large language and vision foundation models — pre-trained on massive datasets using transformer architectures — have revolutionized many fields. It is thus interesting to ask whether a foundation model — subject to suitable data, parameter scaling and training — could enable learned simulations of chemistry and materials. Here, in this study, we review the field of machine-learned interatomic potentials (MLIPs) and posit that scaling up large and diverse chemical and materials datasets and highly expressive architectures using advanced training strategies should result in models that are: more efficient, transferable, robust to out-of-distribution scenarios, and easier to fine-tune to a variety of downstream physical observables than models trained from scratch on small datasets corresponding to specific, targeted atomistic simulation tasks. We provide specific criteria for creating such large-scale MLIP foundation models, coordinated strategies for their development, evaluation and deployment, and highlight potential emergent capabilities that could transform predictive simulations in chemistry and materials science and accelerate discovery across multiple technological domains.

Yuan, Eric C.-Y. [University of California, Berkel↗
DOE OSTI2025

An AI-accelerated pathway for reproducible and stable halide perovskites

Halide perovskites (HPs) have remarkable optoelectronic properties, and in the last decade their photovoltaic power conversion efficiency and light-emitting diode efficiency have skyrocketed. Despite the surge in research on these burgeoning materials, two key challenges in the field remain: material irreproducibility and instability. Their behavior is especially dynamic in response to environmental stressors, due to complex interactions with the perovskite crystal lattice. Here, in this review, we survey the latest achievements in HP materials research accomplished with the assistance of artificial intelligence (AI), through the implementation of automated experimentation and machine learning (ML) data analysis. Automated synthesis and characterization tackle problems with material irreproducibility by systematically controlling parameters with very high precision, creating massive datasets, and allowing methodical comparisons from which unbiased conclusions can be drawn. AI can reveal otherwise unnoticed trends, inform future experiments with the highest potential information gain, and forecast future performance. The review concludes with a forward viewpoint of how human-assisted closed-loop laboratories and shared databases allow halide perovskite materials’ processing, properties, and performance to be potentially optimized with AI, accelerating the development of highly reproducible and stable optoelectronic devices.

Hering, Abigail R. [Univ. of California, Davis, CA↗
DOE OSTI2025

Using the optimal combined index weight ratio to improve the probability of anomaly detection in big area additive manufacturing

Big Area Additive Manufacturing (BAAM) of composites requires significant time, energy, and material, so it is critical to reduce production inefficiencies to make functional parts without multiple iterations. Statistical process control coupled with Principal Component Analysis (PCA) is a powerful technique that provides a quick, computationally inexpensive, and intuitive way for operators to detect defects that form in a manufacturing process without massive datasets. Recently, a combined index that is a weighted sum of the Hotelling's T 2 and squared residual error statistics has been proposed that can be monitored in one chart, improving interpretation accuracy and simplicity. However, the literature does not offer a formal method to optimise the weights. Here, we introduce two new approaches to the traditional weight selection approach using simulated and BAAM image data. Approach 1 uses a theoretically motivated optimum inspired by probabilistic principal component analysis. Approach 2 systematically varies the ratio of the weights to find the optimum. We show that approach 1 delivers optimal anomaly detection performance in select cases while approach 2 fares better in practice. Surprisingly, we also show that choosing a more complex PCA model has a minimal negative impact on anomaly detection performance compared to a more simplistic model.

3-dimensional printing↗
DOE OSTI2025

Scalable edge clustering of dynamic graphs via weighted line graphs

Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.

Data Analysis↗
DOE OSTI2025

Clustering at Massive Scale

ClaMS provides hierarchical clustering technology for use on massive, high-dimensional datasets that require distributed memory for processing. The algorithm employed is inspired by the popular HDBSCAN algorithm but makes use of computational kernels better suited for distributed computing. ClaMS is built on scalable nearest neighbor graph construction, metric forest completion, and approximate minimum spanning tree techniques.

Stanley, ThomasA [Lawrence Livermore National Labo↗
DOE OSTI2026

Feedback, physics, and forecasts: The emerging paradigm of machine learning-driven battery research

Machine learning (ML) is reshaping how we understand, predict, and optimize electrochemical systems. In batteries, ML accelerates discovery across chemistry, design, and operation by transforming massive experimental and simulated datasets into predictive, interpretable models. This review consolidates a decade of progress in ML-driven battery innovation, from early-cycle feature extraction to operando image analysis and physics-informed modeling. We categorize approaches by data domain and physical fidelity, emphasizing interpretable ML for diagnostics, reinforcement learning for control, and multi-objective optimization for lifetime extension strategies. Additionally, we demonstrate how integrated models accelerate discovery, reduce testing time, and guide sustainable design. Economic analyses furthermore illustrate how these advances can lower cost per cycle and improve circularity. Together, these developments chart a path toward self-optimizing, sustainable battery technologies.

artificial intelligence↗
DOE OSTI2024

IMS Rapid Response 2024 Summary Report: A Machine Learning Potential for the Periodic Table

Stockpile stewardship and nuclear waste remediation are inherently chemically complex, involving practically the full diversity of the periodic table, but existing methods are too expensive or not functional for a large diversity of atom types. Overall, the field of machine learning interatomic potentials (MLIPs) has advanced dramatically in 2024 with large high-accuracy datasets existing for bulk, surface, and organic chemical systems and new online leaderboards for diverse chemistry. To participate in, and bring LANL interests into this ecosystem, here, we have built upon existing technologies created by LANL to create a framework capable of creating machine learning interatomic potentials (MLIPs) for over 90 atom types. Our results have created a massively diverse coordination complex training dataset more than 3 times the size of existing datasets, parallelized MLIP training over multiple GPUs, enabling the training of an MLIP spanning the periodic table at 20 times the speed of prior training on 32 GPUs. These advances are substantial towards creation on foundational MLIPs for LANL-specific application areas.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Dataset of simulated vibrational density of states and X-ray diffraction profiles of mechanically deformed and disordered atomic structures in Gold, Iron, Magnesium, and Silicon

This dataset is comprised of a library of atomistic structure files and corresponding X-ray diffraction (XRD) profiles and vibrational density of states (VDoS) profiles for bulk single crystal silicon (Si), gold (Au), magnesium (Mg), and iron (Fe) with and without disorder introduced into the atomic structure and with and without mechanical loading. Included with the atomistic structure files are descriptor files that measure the stress state, phase fractions, and dislocation content of the microstructures. All data was generated via molecular dynamics or molecular statics simulations using the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) code. This dataset can inform the understanding of how local or global changes to a materials microstructure can alter their spectroscopic and diffraction behavior across a variety of initial structure types (cubic diamond, face-centered cubic (FCC), hexagonal close-packed (HCP), and body-centered cubic (BCC) for Si, Au, Mg, and Fe, respectively) and overlapping changes to the microstructure (i.e., both disorder insertion and mechanical loading).

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2025

Cosmological Constraints on Secluded Dark Radiation

Dark radiation (DR) is ubiquitous in physics beyond the Standard Model (SM), and its interactions with the SM and dark matter (DM) lead to a variety of interesting effects on cosmological observables. However, even in scenarios where DR is 'secluded', i.e., only gravitationally interacting with SM and DM, it can leave discernible signatures. We present a comprehensive study of four different types of DR: free-streaming, self-interacting (coupled), decoupling, and recoupling DR, and vary initial conditions to include both adiabatic and isocurvature perturbations. In addition to these properties, we also vary neutrino energy density, DR energy density, and the SM neutrino masses to perform a general analysis and study degeneracies among neutrino and DR properties. We derive constraints using the cosmic microwave background, large-scale structure, and supernova datasets. We find no significant preference for physics beyond the $Λ$CDM model, but data exhibit interesting interplays between different physical quantities. When the neutrino energy density is allowed to vary, we find that the cosmological dataset prefers massless free-streaming DR over massive neutrinos, leading to a significant relaxation of the neutrino mass bound. Although we do not find any evidence of DR isocurvature, the data show support for a strong blue tilt of the isocurvature power spectrum. Our analysis also highlights the degeneracy of various DR parameters with the Hubble constant $H_0$ resulting in a mild relaxation of the $H_0$ tension.

Chang, Jae Hyeok [Fermilab; Illinois U., Chicago] ↗
DOE OSTI2026

Untargeted, tandem mass spectrometry (LC/MS-MS) metaproteomes from soil samples in control and warming plots in Blodgett Forest, CA (2014-2021)

The pathways of carbon transport and loss through and from soils—soil organic matter (SOM) depolymerization to dissolved organic carbon and mineralization to carbon dioxide (CO2)—are fundamentally driven by microbial activity, which is strongly regulated by environmental conditions. As part of Lawrence Berkeley National Laboratory (LBNL) Terrestrial Ecosystem Science (TES) Belowground Biogeochemistry Science Focus Area (SFA), we have established a novel whole-soil long-term warming experiment at the University of California (UC) Blodgett Forest Research Station (Sierra Nevada) in 2014, where we study the role of biogeochemical, microbial and geochemical process interactions in SOM decomposition and stabilization. This package contains soil metaproteomics data in the context of site specific metagenomes from soil depth profiles in three paired control and warming plots from a temperate mixed forest in Northern California. Each paired plot had been subjected to experimental warming since June 2014 to simulate a predicted climate change scenario for northern California. These metaproteomes were collected in 2018 after 4.5 years of warming from five depth intervals (0-10 cm, 10-30 cm, 30-45 cm, 45-60 cm, 60-80 cm). For protein identification, the collected spectra were searched following a target-decoy search strategy against a database of metagenome predicted proteins (covering 96 samples from 2014 to 2021) representing the complete sequence diversity at the site. Data was searched with mass spectrometry database search tool (MS-GF+) using Pacific Northwest National Laboratory (PNNL)'s Data Management System (DMS) Processing pipeline. The metagenomes are published as part of another data package. Raw metaproteomic data and the data products from MS-GF+ are deposited in the Mass Spectrometry Interactive Virtual Environment (MassIVE) database under accession no. MSV000097826. Here we present a dataset that includes spectral counts for the detected proteins across samples (EMSL50964_BrodieAllMAGs_Globals_SC.txt), the sequences of the detected proteins, and sample metadata file that contains site information for the soil metaproteome samples.

Belowground Biogeochemistry Science Focus Area↗
DOE OSTI2025

Tracing the Cosmic Evolution of the Cool Circumgalactic Medium of Luminous Red Galaxies with DESI Year 1 Data

We investigate the properties of the cool circumgalactic medium (CGM) of massive galaxies and their cosmic evolution. By using the year 1 dataset of luminous red galaxies (LRGs) and QSOs from the Dark Energy Spectroscopic Instrument survey, we construct a sample of approximately 600,000 galaxy-quasar pairs and measure the radial distribution and kinematics of the cool gas traced by Mg II absorption lines as a function of galaxy properties from redshift 0.4 to redshift 1.2. Our results show that the covering fraction of the cool gas around LRGs increases with redshift, following a trend similar to the global evolution of galaxy star formation rate. At small radii (< 0.3rvir), the covering fraction anti-correlates with stellar mass, suggesting that mass-dependent processes suppress the cool gas content in the inner region. In addition, we measure the gas dispersion by modeling the velocity distribution of absorbers with a narrow and a broad components -- sigma_n ~ 160 and sigma_b ~ 380 km/s -- and quantify their relative contributions. The results show that the broad component becomes more prominent in the outer region, and its relative importance in the central region grows with increasing stellar mass. Finally, we discuss possible origins of the cool gas around massive galaxies, including the contribution of satellite galaxies and the precipitation scenario.

Chang, Yu-Ling [Taiwan, Natl. Taiwan U.] (ORCID:00↗
DOE OSTI2025

Developing reliable machine learning interatomic potential for Fe–Cr–Ni austenitic alloys

Gaining atomistic understanding of mechanical behavior of heat-resistant structural materials such as Fe–Cr–Ni-based alloys requires an approach with an accuracy close to density functional theory (DFT) that considers the intrinsic properties of the bulk lattice and important defects such as stacking faults, grain boundaries, and surfaces. This work aims to develop reliable machine learning interatomic potential (MLIAP) at cross-scale for Fe–Cr–Ni ternary alloys with a focus on the face-centered-cubic (fcc) solid solution structure. Leveraging the advantages of moment tensor potentials, which typically necessitate a relatively small training dataset and enable rapid calculations using the large-scale atomic/molecular massively parallel simulator package, we ensure the stability and accuracy of the trained potentials. Important defects such as stacking faults, grain boundaries, and surfaces for wide-range compositions are investigated. Structural, thermal, elastic, and defect properties are determined from molecular dynamics simulations comprising several thousand atoms, generated via canonical Monte Carlo simulations guided by the trained potential. The trained potential allows efficient atomic simulations of structural, thermal, and mechanical properties of fcc Fe–Cr–Ni solid solution alloys as a function of composition and temperature. Therefore, the MLIAP approach represents a major advancement from DFT calculations that are limited to small simulation sizes and traditional molecular dynamics simulations using relatively low accuracy potentials. Furthermore, this work outlines a practical foundation for further investigating the structural evolution and mechanical behavior of austenitic stainless steel and nickel-based alloys in a wide array of applications in extreme environments.

Crystal structure↗
DOE OSTI2025

Atacama Cosmology Telescope: Multiprobe cosmology with unWISE galaxies and ACT DR6 CMB lensing

We present a joint analysis of the cosmic microwave background (CMB) lensing power spectra measured from the Data Release 6 of the Atacama Cosmology Telescope (ACT) and Planck PR4, cross-correlations between the ACT and Planck lensing reconstruction and galaxy clustering from unWISE, and the unWISE clustering auto-spectrum. We obtain 1.5% constraints on the matter density fluctuations at late times parametrized by the best constrained parameter combination 𝑆$^{3⁢x⁢2⁢pt}_{8}$ ≡ 𝜎 8 ⁢(Ω 𝑚 /0.3) 0.4 = 0.815 ± 0.012. The commonly used 𝑆 8 ≡ 𝜎 8⁢ (Ω 𝑚 /0.3) 0.5 parameter is constrained to 𝑆 8 = 0.816 ± 0.015. In combination with baryon acoustic oscillation (BAO) measurements we find 𝜎 8 = 0.815 ± 0.012. We also present sound-horizon-independent estimates of the present day Hubble rate of 𝐻 0 = 66.4$^{+3.2}_{−3.7}$ km s −1 Mpc −1 from our large scale structure data alone and 𝐻 0 = 64.3$^{+2.1}_{−2.4}$ km s −1 Mpc −1 in combination with uncalibrated supernovae from Pantheon+. Using parametric estimates of the evolution of matter density fluctuations, we place constraints on cosmic structure in a range of high redshifts typically inaccessible with cross-correlation analyses. Combining lensing cross- and autocorrelations, we derive a 3.3% constraint on the integrated matter density fluctuations above 𝑧 = 2.4, one of the tightest constraints in this redshift range and fully consistent with a Λ cold dark matter (Λ⁢CDM) model fit to the primary CMB from Planck. Finally, combining with primary CMB observations and using the extended low redshift coverage of these combined datasets we derive constraints on a variety of extensions to the Λ⁢CDM model including massive neutrinos, spatial curvature, and dark energy. We find in flat Λ⁢CDM⁢ ∑𝑚 𝜈 < 0.12 eV at 95% confidence using the large scale structure data, BAO measurements from Sloan Digital Sky Survey, and primary CMB observations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

TxDOT Road Elevation Model Dataset

This dataset provides three formats of Road Elevation Model (REM) data: 3D road line/polygon GeoPackage (GPKG), road lidar LAZ and COPC LAZ, and road digital surface model (DSM) GeoTIFF. Data are produced from the ~50TB TxGIO (formerly TNRIS) state lidar collections. This dataset is currently organized by maintenance section in each TxDOT district. Computation is done on GPU computing resources at Oak Ridge National Laboratory (ORNL), through a Strategic Partnership Project with UT Austin and an NSF ACCESS computing allocation award that enables fast massive data movement between TACC Corral and ORNL CADES/OLCF using Globus. In addition to this release from ORNL, a copy of this dataset can also be downloaded at https://web.corral.tacc.utexas.edu/nfiedata/road3d/.

13 HYDRO ENERGY↗