Data-Driven Closure Modeling in Sparsely Observed Dynamical Systems
Our collaborators at Texas A&M will be presenting this work.
Drennan, Daniel [Texas A&M University]↗
SEARCH · Engineering Papers
Results for “Sparse Data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Rare earth element enrichment in coal and coal-adjacent strata of the Uinta Region, Utah and Colorado
This study aims to quantify rare earth element enrichment within coal and coal-adjacent strata in the Uinta Region of central Utah and western Colorado. Rare earth elements are a subset of critical minerals as defined by the U.S. Geological Survey. These elements are used for a wide variety of applications, including renewable energy technology in the transition toward carbon-neutral energy. While rare earth element enrichment has been associated with Appalachian coals, there has been a more limited evaluation of western U.S. coals. Here, samples from six active mines, four idle/historical mines, four mine waste piles, and seven stratigraphically complete cores within the Uinta Region were geochemically evaluated using portable X-ray fluorescence ( n = 3,113) and inductively coupled plasma-mass spectrometry ( n = 145) elemental analytical methods. Results suggest that 24%–45% of stratigraphically coal-adjacent carbonaceous shale and siltstone units show rare earth element enrichment (>200 ppm), as do 100% of sampled igneous material. A small subset (5%–8%) of coal samples display rare earth element enrichment, specifically in cases containing volcanic ash. This study proposes two multi-step depositional and diagenetic models to explain the enrichment process, requiring the emplacement and mobilization of rare earth element source material due to hydrothermal and other external influences. Historical geochemical evaluations of Uinta Region coal and coal-adjacent data are sparse, emphasizing the statistical significance of this research. These results support the utilization of active mines and coal processing waste piles for the future of domestic rare earth element extraction, offering economic and environmental solutions to pressing global demands.
Coe, Haley H.↗
A numerical framework for simulating the atmospheric variability of supermicron marine biogenic ice nucleating particles
Abstract. We present a framework for estimating concentrations of episodically elevated high-temperature marine ice nucleating particles (INPs) in the sea surface microlayer and their subsequent emission into the atmospheric boundary layer. These episodic INPs have been observed in multiple ship-based and coastal field campaigns, but the processes controlling their ocean concentrations and transfer to the atmosphere are not yet fully understood. We use a combination of empirical constraints and simulation outputs from an Earth system model to explore different hypotheses for explaining the variability of INP concentrations, and the occurrence of episodic INPs, in the marine atmosphere. In our calculations, we examine the following two proposed oceanic sources of high-temperature INPs: heterotrophic bacteria and marine biopolymer aggregates (MBPAs). Furthermore, we assume that the emission of these INPs is determined by the production of supermicron sea spray aerosol formed from jet drops, with an entrainment probability that is described by Poisson statistics. The concentration of jet drops is derived from the number concentration of supermicron sea spray aerosol calculated from model runs. We then derive the resulting number concentrations of marine high-temperature INPs (at 253 K) in the atmospheric boundary layer and compare their variability to atmospheric observations of INP variability. Specifically, we compare against concentrations of episodically occurring high-temperature INPs observed during field campaigns in the Southern Ocean, the Equatorial Pacific, and the North Atlantic. In this case study, we evaluate our framework at 253 K because reliable observational data at this temperature are available across three different ocean regions, but suitable data are sparse at higher temperatures. We find that heterotrophic bacteria and MBPAs acting as INPs provide only a partial explanation for the observed high INP concentrations. We note, however, that there are still substantial knowledge gaps, particularly concerning the identity of the oceanic INPs contributing most frequently to episodic high-temperature INPs, their specific ice nucleation activity, and the enrichment of their concentrations during the sea–air transfer process. Therefore, targeted measurements investigating the composition of these marine INPs and drivers for their emissions are needed, ideally in combination with modeling studies focused on the potential cloud impacts of these high-temperature INPs.
54 ENVIRONMENTAL SCIENCES↗
Conservative Estimation of Tail Probabilities from Limited Sample Data
Several sparse-sample uncertainty quantification (UQ) methods are compared for conservative but not overly conservative estimation of small tail probabilities involving responses that lay beyond specified thresholds in the tails of probability distributions. Sixteen very differently shaped distributions (or probability density functions, PDFs) and tail probability magnitudes ranging from 10 -5 to 10 -1 are considered in order for the study to be relevant to a wide range of risk analysis and quantification of margins and uncertainty (QMU) problems. The emphasis of the study is on limited data regimes ranging from N = 2 to 20 samples, reflective of most experimental and some expensive computational situations. Relatively simple sparse-sample UQ methods tested for this regime involve statistical tolerance interval "Equivalent Normal and related "Ensemble of Normals" and "Superdistribution (SD) approaches. (The independently derived SD is effectively equivalent to the Bayesian posterior predictive distribution given the assumptions of the derivation.) The performance of the methods was generally improved for N ≥ 5 samples with a generalized Jackknife resampling technique, which determines a tail probability estimate by averaging estimates from smaller sub-samples. Several quantitative metrics for method conservatism and accuracy of tail probability estimation are used to assess and rank the methods' performance over many random trials for each test PDF and probability magnitude. A variant of Bootstrap resampling was also tried, but did not significantly improve tail probability estimates in most cases. Detailed results are presented from over 100-million tests over the above factors that provide useful granular information on which methods or combination of methods perform best in various areas of the factor space.
97 MATHEMATICS AND COMPUTING↗
3D Geologic Framework Modelling of the Los Alamos National Laboratory Site and Pajarito Plateau: Integrating a realistic 3D fault network and modelling subsurface relationships in a sparsely sampled and complex geologic region
The subsurface geology beneath the Pajarito Plateau is critical to understanding the seismic hazard of the Pajarito Fault System, yet our understanding of this geology is relatively poor. While previous 3D geologic framework models of the area have been created for the purposes of understanding hydrogeologic flow, they are inadequate for the purposes of understanding the Pajarito Fault System. The specific challenges of using oil and gas software for this purpose include: (1) the geologic complexities resulting from volcanism and tectonism; (2) a need for a high level of stratigraphic detail over a large area; (3) a near complete lack of seismic data; and (4) sparse wellbore data. Presented here is a workflow that handles these challenges of adapting commercially available software used by the oil and gas industries to this seismic hazard problem.
58 GEOSCIENCES↗
1-D coupled surface flow and transport equations revisited via the physics-informed neural network approach
The de Saint-Venant equation (SVE) and advection–diffusion equation (ADE) are commonly employed to solve solute transport problems in surface water. In this work, we propose a mesh-free method based on the physics-informed neural network (PINN) to solve the one dimensional (1-D) SVE, ADE, and the coupled SVE and ADE (SVE-ADE) under various initial and boundary conditions. The PINN model extends the architecture of deep neural networks (DNNs) with implementation of loss function, which are additionally subject to constraints imposed by the physical laws of SVE and ADE, along with their initial and boundary conditions. In such a manner, PINNs can be quickly steered to the true solution while obeying the physical laws. The results of PINN model are compared with the analytical and/or numerical solutions under various conditions to investigate its accuracy and efficiency in solving the SVE, ADE, and SVE-ADE. Our results indicate PINN can accurately simulate the shock wave morphology and avoid numerical dissipation in unsteady flow condition. The PINN method outweighed traditional numerical methods in several aspects, including its ability to function with small amounts of data, no grid discretization, and random selection of sampling points, etc. Additionally, the PINN method is also suitable for solving inverse problems with sparse and noisy data. With 1% noise and 2000 initial and boundary condition points (N u ), the errors of the estimated flow rate (v) and diffusion coefficient (D) are 0.003% and 0.105%, respectively, which indicate the accuracy and robustness of the proposed method. Finally, our results indicate the capability and robustness of the proposed PINN methodology for solving multi-physics problems, irrespective of the presence of sparse and noisy data in the training dataset.
54 ENVIRONMENTAL SCIENCES↗
Towards a self-consistent model of the convective core boundary in upper main sequence stars: I. 2.5D and 3D simulations
There is strong observational evidence that the convective cores of intermediate-mass and massive main sequence stars are substantially larger than those predicted by standard stellar-evolution models. However, it is unclear what physical processes cause this phenomenon or how to predict the extent and stratification of stellar convective boundary layers. Convective penetration is a thermal-timescale process that is likely to be particularly relevant during the slow evolution on the main sequence. We use our low-Mach-number S EVEN -L EAGUE H YDRO code to study this process in 2.5D and 3D geometries. Starting with a chemically homogeneous model of a 15 M ⊙ zero-age main sequence star, we construct a series of simulations with the luminosity increased and opacity decreased by the same factor, ranging from 10 3 to 10 6 . After reaching thermal equilibrium, all of our models show a clear penetration layer; its thickness becomes statistically constant in time and it is shown to converge upon grid refinement. The penetration layer becomes nearly adiabatic with a steep transition to a radiative stratification in simulations at the lower end of our luminosity range. This structure corresponds to the adiabatic ‘step overshoot’ model often employed in stellar-evolution calculations. The simulations with the highest and lowest luminosity differ by less than a factor of two in the penetration distance. The high computational cost of 3D simulations makes our current 3D data set rather sparse. Depending on how we extrapolate the 3D data to the actual luminosity of the initial stellar model, we obtain penetration distances ranging from 0.09 to 0.44 pressure scale heights, which is broadly compatible with observations.
79 ASTRONOMY AND ASTROPHYSICS↗
Reducing Communication in Graph Neural Network Training
Graph Neural Networks (GNNs) are powerful and flexible neural networks that use the naturally sparse connectivity information of the data. GNNs represent this connectivity as sparse matrices, which have lower arithmetic intensity and thus higher communication costs compared to dense matrices, making GNNs harder to scale to high concurrencies than convolutional or fully-connected neural networks. Here, we introduce a family of parallel algorithms for training GNNs and show that they can asymptotically reduce communication compared to previous parallel GNN training methods. We implement these algorithms, which are based on 1D, 1. 5D, 2D, and 3D sparse-dense matrix multiplication, using torch.distributed on GPU-equipped clusters. Our algorithms optimize communication across the full GNN training pipeline. We train GNNs on over a hundred GPUs on multiple datasets, including a protein network with over a billion edges.
97 MATHEMATICS AND COMPUTING↗
Testing to Evaluate Processes Expected to Occur during MSR Salt Spill Accidents
Obtaining a license for a new nuclear reactor requires the identification and assessment of the potential consequences of specified accident scenarios, which are achieved using accident progression models. Those models need to be parameterized and validated using experimental data, but existing experimental data addressing processes relevant to molten salt reactor accidents are sparse. Specifically, experimental data that quantify the sensitives of important processes to the initial conditions of the spill, the ambient environment, and the containment features are needed to parameterize individual process models. Integrated experiments that simulate accident scenarios are also needed to provide data for model validation, but these experiments will require the use of proven methods to quantify the processes under evaluation. The overarching objectives of this work are to develop the methods for simulating the targeted processes, to determine the effectiveness of the methods in producing the data required for model development, to generate data that can be used to parameterize individual process models, and to provide key insights into the behavior of spilled molten salt that should be considered in models. Experimental methods were designed to quantify aspects of individual processes expected to occur during or after a molten salt spill accident that will affect the fate of spilled molten salt and the radionuclides within. These processes include 1) molten salt spreading and heat transfer, 2) molten salt flowing and freezing in tubing, 3) stainless steel corrosion kinetics in molten salt, and 4) molten salt splashing and aerosol generation. The initial tests described in this report were conducted using eutectic FLiNaK to demonstrate the test methods, the data that are generated, and the analyses of the data to derive values needed for modeling. The primary variables that were tested include initial salt temperature and the presence of volatile surrogate fission products (e.g., cesium and iodine). The developed methods are shown to be effective in quantifying the desired processes and can be applied to study more complex salt compositions of interest to molten salt reactor developers, a wide range of environmental conditions of interest to modelers, and additional variables relevant to salt spill accidents. The developed methods and insights gained from laboratory tests can also be incorporated in future large-scale integrated tests used to simulate molten salt spill accidents.
21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
powersqueeze
powersqueeze (psqz) is a truncated power iteration library intended for high-performance computing platforms. psqz efficiently produces low-dimensional, linear measurements of graph matrix spectra by combining classical power iteration with sparse Johnson-Lindenstrauss transforms. psqz is intended to produce high-quality, fast, data-oblivious low-dimensional representations of high-dimensional sparse data such as graphs and term-document matrices. psqz is intended to replace similar workflows that depend on directly approximating a truncated eigendecomposition (e.g., the first step of spectral clustering), which is a much more expensive operation.
Priest, BenjaminW [Lawrence Livermore National Lab↗
Phase curves of small bodies from the SLOAN Moving Objects Catalog
Extensive photometric surveys continue to produce enormous stores of data on small bodies. These data are typically sparsely obtained at arbitrary (or unknown) rotational phases. Therefore, new methods for processing such data need to be developed to make the most of these vast catalogs. We aim to produce a method of recreating the phase curves of small bodies by considering the uncertainties introduced by the nominal errors in the magnitudes and the effect introduced by rotational variations. Here, we use the SLOAN Moving Objects Catalog data as a benchmark to construct phase curves of all small bodies in u', g', r', i', and z' filters. From the phase curves, we obtain the absolute magnitudes and we use them to set up the absolute colors, which are the colors of the asteroids that are not affected by changes in the phase angle. We selected objects with ≥3 observations taken in at least one filter and spanning over a minimum of 5 degrees in the phase angle. We developed a method that combines Monte Carlo simulations and Bayesian inference to estimate the absolute magnitudes using the HG 12 * photometric system. We obtained almost 15 000 phase curves, with about 12 000 of these including all five filters. The absolute magnitudes and absolute colors are compatible with previously published data that support our method. The method we developed is fully automatic and well suited for a run based on large amounts of data. Moreover, it includes the nominal uncertainties in the magnitudes and the whole distribution of possible rotational states of the objects producing what are possibly less precise values, that is, larger uncertainties, but more accurate, namely, closer to the actual value. To our knowledge, this work is the first to include the effect of rotational variations in such a manner.
79 ASTRONOMY AND ASTROPHYSICS↗
Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data
High-energy large-scale particle colliders produce data at high speed in the order of 1 terabytes per second in nuclear physics and petabytes per second in high energy physics. Developing real-time data compression algorithms to reduce such data at high throughput to fit permanent storage has drawn increasing attention. Specifically, at the newly constructed sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC), a time projection chamber is used as the main tracking detector, which records particle trajectories in a volume of three-dimensional (3D) cylinder. The resulting data are usually very sparse with occupancy around 10.8%. Such sparsity presents a challenge to conventional learning-free lossy compression algorithms, such as SZ, ZFP, and MGARD. The 3D convolutional neural network (CNN)-based approach, Bicephalous Convolutional Autoencoder (BCAE), outperforms traditional methods both in compression rate and reconstruction accuracy. BCAE can also utilize the computation power of graphical processing units suitable for deployment in a modern heterogeneous highperformance computing environment. This work introduces two BCAE variants: BCAE++ and BCAE-2D. BCAE++ achieves a 15% better compression ratio and a 77% better reconstruction accuracy measured in mean absolute error compared with BCAE. BCAE-2D treats the radial direction as the channel dimension of an image, resulting in a 3× speedup in compression throughput. In addition, we demonstrate an unbalanced autoencoder with a larger decoder can improve reconstruction accuracy without significantly sacrificing throughput. Lastly, we observe both the BCAE++ and BCAE-2D can benefit more from using half-precision mode in throughput (76 - 79% increase) without loss in reconstruction accuracy. The source code and links to data and pretrained models can be found at https://github.com/BNL-DAQ-LDRD/NeuralCompression_v2
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Scalable Incremental Checkpointing using GPU-Accelerated De-Duplication
Writing large amounts of data concurrently to stable storage is a typical I/O pattern of many HPC workflows. This pattern introduces high I/O overheads and results in increased storage space utilization especially for workflows that need to capture the evolution of data structures with high frequency as checkpoints. In this context, many applications, such as graph pattern matching, perform sparse updates to large data structures between checkpoints. For these applications, incremental checkpointing techniques that save only the differences from one checkpoint to another can dramatically reduce the checkpoint sizes, I/O bottlenecks, and storage space utilization. However, such techniques are not without challenges: it is non-trivial to transparently determine what data has changed since a previous checkpoint and assemble the differences in a compact fashion that does not result in excessive metadata. State-of-art data reduction techniques (e.g., compression and de-duplication) have significant limitations when applied to modern HPC applications that leverage GPUs: slow at detecting the differences, generate a large amount of metadata to keep track of the differences, and ignore crucial spatiotemporal checkpoint data redundancy. This paper addresses these challenges by proposing a Merkle tree-based incremental checkpointing method to exploit GPUs' high memory bandwidth and massive parallelism. Experimental results at scale show a significant reduction of the I/O overhead and space utilization of checkpointing compared with state-of-the-art incremental checkpointing and compression techniques.
Tan, Nigel↗
Deep structural clustering for single-cell RNA-seq data jointly through autoencoder and graph neural network
Abstract Single-cell RNA sequencing (scRNA-seq) permits researchers to study the complex mechanisms of cell heterogeneity and diversity. Unsupervised clustering is of central importance for the analysis of the scRNA-seq data, as it can be used to identify putative cell types. However, due to noise impacts, high dimensionality and pervasive dropout events, clustering analysis of scRNA-seq data remains a computational challenge. Here, we propose a new deep structural clustering method for scRNA-seq data, named scDSC, which integrate the structural information into deep clustering of single cells. The proposed scDSC consists of a Zero-Inflated Negative Binomial (ZINB) model-based autoencoder, a graph neural network (GNN) module and a mutual-supervised module. To learn the data representation from the sparse and zero-inflated scRNA-seq data, we add a ZINB model to the basic autoencoder. The GNN module is introduced to capture the structural information among cells. By joining the ZINB-based autoencoder with the GNN module, the model transfers the data representation learned by autoencoder to the corresponding GNN layer. Furthermore, we adopt a mutual supervised strategy to unify these two different deep neural architectures and to guide the clustering task. Extensive experimental results on six real scRNA-seq datasets demonstrate that scDSC outperforms state-of-the-art methods in terms of clustering accuracy and scalability. Our method scDSC is implemented in Python using the Pytorch machine-learning library, and it is freely available at https://github.com/DHUDBlab/scDSC.
Gan, Yanglan↗
Accelerating GNNs on GPU Sparse Tensor Cores through N:M Sparsity-Oriented Graph Reordering
Recent advancements in GPU hardware support have introduced the capability to leverage N:M sparse patterns for substantial performance gains. Graphs in Graph Neural Networks (GNNs) are typically sparse, but the sparsity is often irregular, not conforming to such sparse patterns. In this paper, we propose a novel graph reordering algorithm, the first of its kind, to reshape irregular graph data into the N:M structured sparse pattern at the tile level, allowing linear-algebra-based graph operations in GNNs to benefit from the N:M sparse hardware. The optimization is lossless, maintaining the accuracy of GNN. It can remove 98-100\% violations of the N:M sparse patterns at the vector level, and increase the proportion of conforming graphs in SuiteSparse collection from 5-9\% to 88.7-93.5\%. On A100 GPUs, the optimization accelerates Sparse Matrix Matrix (SpMM) by up to 43X (2.3X -- 7.5X on average) and speeds up the key graph operations in GNNs on real graphs by as much as 8.6X (3.5X on average).
artificial intelligence, graph neural networks↗
Prong Segmentation using Point Set Transformers in Multiple View Neutrino Detectors
NOvA is a long-baseline neutrino experiment studying neutrino oscillations by detecting neutrinos from the NuMI beam at Fermilab. Its physics analysis relies on accurate prong segmentation, which involves matching each hit to its source particle and identifying the particle type. This task has commonly been addressed using a combination of traditional clustering algorithms and convolutional neural networks (CNNs). However, NOvA’s detector design presents data as two sparse and decoupled 2D images (XZ and YZ views) rather than a native 3D representation, posing a significant challenge for traditional CNN-based models. In this talk, we propose a novel neural network based on the Point Set Transformer. By treating detector hits as sparse point clouds and implementing a cross-view attention mechanism, our model enables efficient information mixing between both views. Evaluated on NOvA simulated data, our model achieves superior accuracy while requiring significantly fewer computational resources compared to other models. Furthermore, the model demonstrates great performance when applied to Liquid Argon Time Projection Chamber (LArTPC) data, which shows its potential as a universal prong segmentation algorithm for multiple view neutrino detectors.
Liu, Jiaxi [UC, Irvine]↗
Poisson hurdle model-based method for clustering microbiome features
Abstract Motivation High-throughput sequencing technologies have greatly facilitated microbiome research and have generated a large volume of microbiome data with the potential to answer key questions regarding microbiome assembly, structure and function. Cluster analysis aims to group features that behave similarly across treatments, and such grouping helps to highlight the functional relationships among features and may provide biological insights into microbiome networks. However, clustering microbiome data are challenging due to the sparsity and high dimensionality. Results We propose a model-based clustering method based on Poisson hurdle models for sparse microbiome count data. We describe an expectation–maximization algorithm and a modified version using simulated annealing to conduct the cluster analysis. Moreover, we provide algorithms for initialization and choosing the number of clusters. Simulation results demonstrate that our proposed methods provide better clustering results than alternative methods under a variety of settings. We also apply the proposed method to a sorghum rhizosphere microbiome dataset that results in interesting biological findings. Availability and implementation R package is freely available for download at https://cran.r-project.org/package=PHclust. Supplementary information Supplementary data are available at Bioinformatics online.
59 BASIC BIOLOGICAL SCIENCES↗
Machine Learning Based Metamodel for Faster Life Cycle Assessment of Large Portfolio of Buildings
Managing a large portfolio of buildings involves decisions on reuse, retrofit, renovation, rehabilitation, and new construction, influenced by trade-offs between performance metrics such as cost, time, and operational flexibility over the building's life cycle. Traditional life cycle assessment tools for evaluating these metrics can be labor- and compute-intensive, requiring extensive data and modeling for each building. Metamodels (or surrogate models) using machine learning have been explored as faster alternatives, but training these models has been hindered by the limited availability of comprehensive data on key life cycle metrics. Recent advancements in machine learning, particularly deep learning techniques like zero-shot and few-shot learning, allow models to learn from sparse or limited data. We propose a machine learning-based metamodel that leverages these techniques for rapid estimation of key building life cycle metrics. This presentation will cover the model architecture, data collection, training, and validation processes, along with an ongoing case study applied to a large portfolio of buildings. We will discuss the model's performance in terms of accuracy, compute time, limitations, and its potential for expanding to additional life cycle metrics. This data-driven approach offers a promising direction for the rapid evaluation of large building portfolios.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗