Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Manifold Learning: What, How, and Why

Manifold learning (ML), also known as nonlinear dimension reduction, is a set of methods to find the low-dimensional structure of data. Dimension reduction for large, high-dimensional data is not merely a way to reduce the data; the new representations and descriptors obtained by ML reveal the geometric shape of high-dimensional point clouds and allow one to visualize, denoise, and interpret them. This review presents the underlying principles of ML, its representative methods, and their statistical foundations, all from a practicing statistician's perspective. It describes the trade-offs and what theory tells us about the parameter and algorithmic choices we make in order to obtain reliable conclusions.

Mathematics↗
DOE OSTI2025

Statistical relationships across epigenomes using large-scale hierarchical clustering

Recent advances in genomics and sequencing platforms have revolutionized our ability to create immense data sets, particularly for studying epigenetic regulation of gene expression. However, the avalanche of epigenomic data is difficult to parse for biological interpretation given nonlinear complex patterns and relationships. This attractive challenge in epigenomic data lends itself to machine learning for discerning infectivity and susceptibility. In this study, we explore over 3000 epigenomes of uninfected individuals and provide a framework to characterize the relationships among epigenetic modifiers, their modifiers, genetic loci, and specific immune cell types across all chromosomes using hierarchical clustering. Hierarchical clustering of epigenomic data revealed consistent epigenetic patterns across chromosomes, demonstrating that variation due to epigenetic modifiers is greater than variation between cell types. Gene Ontology and KEGG pathway analyses indicated significant enrichment of genes involved in chromatin remodeling, mRNA splicing, immune responses, and the regulation of microRNAs and snoRNAs. Epigenetic modifiers frequently formed biologically relevant clusters, including the cohesin complex, RNA Polymerase II transcription factors, and PRC2 complex members. These clustering behaviors remained consistent across all chromosomes, supported by entropy analysis and high Adjusted Rand Index scores, indicating robust cross-chromosomal similarity. Co-occurrence analysis further revealed specific sets of modifiers that consistently appeared together within clusters, reflecting shared biological functions and interactions. Validation using another dataset confirmed the reproducibility of these clustering patterns and modifier co-occurrence relationships, underscoring the reliability and generalizability of the methodology.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Realistic galaxy image simulation via score-based generative models

ABSTRACT We show that a denoising diffusion probabilistic model (DDPM), a class of score-based generative model, can be used to produce realistic mock images that mimic observations of galaxies. Our method is tested with Dark Energy Spectroscopic Instrument (DESI) grz imaging of galaxies from the Photometry and Rotation curve OBservations from Extragalactic Surveys (PROBES) sample and galaxies selected from the Sloan Digital Sky Survey. Subjectively, the generated galaxies are highly realistic when compared with samples from the real data set. We quantify the similarity by borrowing from the deep generative learning literature, using the ‘Fréchet inception distance’ to test for subjective and morphological similarity. We also introduce the ‘synthetic galaxy distance’ metric to compare the emergent physical properties (such as total magnitude, colour, and half-light radius) of a ground truth parent and synthesized child data set. We argue that the DDPM approach produces sharper and more realistic images than other generative methods such as adversarial networks (with the downside of more costly inference), and could be used to produce large samples of synthetic observations tailored to a specific imaging survey. We demonstrate two potential uses of the DDPM: (1) accurate inpainting of occluded data, such as satellite trails, and (2) domain transfer, where new input images can be processed to mimic the properties of the DDPM training set. Here we ‘DESI-fy’ cartoon images as a proof of concept for domain transfer. Finally, we suggest potential applications for score-based approaches that could motivate further research on this topic within the astronomical community.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Surrogate models for plasma displacement and current in 3D perturbed magnetohydrodynamic equilibria in tokamaks

Abstract A numerical database of over one thousand perturbed three-dimensional (3D) equilibria has been generated, constructed based on the MARS-F (Liu et al 2000 Phys. Plasmas 7 3681) computed plasma response to the externally applied 3D field sources in multiple tokamak devices. Perturbed 3D equilibria with the n = 1–4 ( n is the toroidal mode number) toroidal periodicity are computed. Surrogate models are created for the computed perturbed 3D equilibrium utilizing model order reduction (MOR) techniques. In particular, retaining the first few eigenstates from the singular value decomposition (SVD) of the data is found to produce reasonably accurate MOR-representations for the key perturbed quantities, such as the perturbed parallel plasma current density and the plasma radial displacement. SVD also helps to reveal the core versus edge plasma response to the applied 3D field. For the database covering the conventional aspect ratio devices, about 95% of data can be represented by the truncated SVD-series with inclusion of only the first five eigenstates, achieving a relative error (RE) below 20%. The MOR-data is further utilized to train neural networks (NNs) to enable fast reconstruction of perturbed 3D equilibria, based on the two-dimensional equilibrium input and the 3D source field. The best NN-training is achieved for the MOR-data obtained with a global SVD approach, where the full set of samples used for NN training and testing are stretched and form a large matrix which is then subject to SVD. The fully connected multi-layer perceptron, with one or two hidden layers, can be trained to predict the MOR-data with less than 10% RE. As a key insight, a better strategy is to train separate NNs for the plasma response fields with different toroidal mode numbers. It is also better to apply MOR and to subsequently train NNs separately for conventional and low aspect ratio devices, due to enhanced toroidal coupling of Fourier spectra in the plasma response in the latter case.

3D equilibrium↗
DOE OSTI2022

The SENSEI Generic In Situ Interface: Tool and Processing Portability at Scale [Book Chapter]

One key challenge when doing in situ processing is the investment required to add code to numerical simulations needed to take advantage of in situ processing. Such instrumentation code is often specialized, and tailored to a specific in situ method or infrastructure. Then, if a simulation wants to use other in situ tools, each of which has its own bespoke API [4], then the simulation code team will quickly become overwhelmed with having a different set of instrumentation APIs, one per in situ tool or method. In an ideal situation, such instrumentation need happen only once, and then the instrumentation API provides access to a large diversity of tools. In this way, a data producer’s instrumentation need not be modified if the user desires to take advantage of a different set of in situ tools. The SENSEI generic in situ interface addresses this challenge, which means that SENSEI-instrumented codes enjoy the benefit of being able to use a diversity of tools at scale, tools that include Libsim, Catalyst, Ascent, as well as user-defined methods written in C++ or Python. SENSEI has been shown to scale to greater than 1M-way concurrency on HPC platforms, and provides support for a rich and diverse collection of common scientific data models. Furthermore, this chapter presents the key design challenges that enable tool and processing portability at scale, some performance analysis, and example science applications of the methods.

Bethel, E. Wes↗
DOE OSTI2025

Search for heavy long-lived charged particles with large ionization energy loss in proton-proton collisions at $$ \sqrt{s} $$ = 13 TeV

Abstract A search for heavy, long-lived, charged particles with large ionization energy loss within the silicon tracker of the CMS experiment is presented. A data set of proton-proton collisions at a center of mass energy at $$ \sqrt{s} $$ s = 13 TeV, collected in 2017 and 2018 at the CERN LHC, corresponding to an integrated luminosity of 101 fb −1, is used in this analysis. Two different approaches for the search are taken. A new method exploits the independence of the silicon pixel and strips measurements, while the second method improves on previous techniques using ionization to determine a mass selection. No significant excess of events above the background expectation is observed. The results are interpreted in the context of the pair production of supersymmetric particles, namely gluinos, top squarks, and tau sleptons, and of the Drell-Yan pair production of fourth generation (τ′) leptons with an electric charge equal to or twice the absolute value of the electron charge (e). An interpretation of a Z’ boson decaying to two τ′ leptons with an electric charge equal to 2e is presented for the first time. The 95% confidence upper limits on the production cross section are extracted for each of these hypothetical particles.

Hayrapetyan, A. [Yerevan Physics Institute]↗
DOE OSTI2025

Search for heavy long-lived charged particles with large ionization energy loss in proton-proton collisions at $\sqrt{s}$ = 13 TeV

A search for heavy, long-lived, charged particles with large ionization energy loss within the silicon tracker of the CMS experiment is presented. A data set of proton-proton collisions at a center of mass energy at $\sqrt{s}$ = 13 TeV, collected in 2017 and 2018 at the CERN LHC, corresponding to an integrated luminosity of 101 fb −1 , is used in this analysis. Two different approaches for the search are taken. A new method exploits the independence of the silicon pixel and strips measurements, while the second method improves on previous techniques using ionization to determine a mass selection. No significant excess of events above the background expectation is observed. The results are interpreted in the context of the pair production of supersymmetric particles, namely gluinos, top squarks, and tau sleptons, and of the Drell-Yan pair production of fourth generation (τ′) leptons with an electric charge equal to or twice the absolute value of the electron charge (e). An interpretation of a Z’ boson decaying to two τ′ leptons with an electric charge equal to 2e is presented for the first time. The 95% confidence upper limits on the production cross section are extracted for each of these hypothetical particles.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

E3SM‐Arctic: Regionally Refined Coupled Model for Advanced Understanding of Arctic Systems Interactions

Earth system models are essential tools for climate projections, but coarse resolutions limit regional accuracy, especially in the Arctic. Regionally refined meshes (RRMs) enhance resolution in key areas while maintaining computational efficiency. This paper provides an overview of the United States (U.S.) Department of Energy's (DOE's) Energy Exascale Earth System Model version 2.1 with an Arctic RRM, hereafter referred to as E3SMv2.1-Arctic, for the atmosphere (25 km), land (25 km), and ocean/ice (10 km) components. We evaluate the atmospheric component and its interactions with land, ocean, and cryosphere by comparing the RRM (E3SM2.1-Arctic) historical simulations (1950–2014) with the uniform low-resolution (LR) counterpart, reanalysis products, and observational data sets. The RRM generally reduces biases in the LR model, improving simulations of Arctic large-scale mean fields, such as precipitation, atmospheric circulation, clouds, atmospheric river frequency, and sea ice thickness. However, it introduces a seasonally dependent surface air temperature bias, reducing the LR cold bias in summer but enhancing the LR warm bias in winter, which contributes to the underestimated winter sea ice area and volume. Radiative feedback analysis shows similar climate feedback strengths in both model configurations, with the RRM exhibiting a more positive surface albedo feedback and contributing to a stronger surface warming than LR. These findings underscore the importance of high-resolution modeling for advancing our understanding of Arctic climate changes and their broader global impacts, although some persistent biases appear to be independent of model resolution at 10–100 km scales.

Energy Exascale Earth System Model (E3SM)↗
DOE OSTI2020

Improved modeling of β electronic recoils in liquid xenon using LUX calibration data

We report here methods and techniques for creating and improving a model that reproduces the scintillation and ionization response of a dual-phase liquid and gaseous xenon time-projection chamber. Starting with the recent release of the Noble Element Simulation Technique (NEST v2.0), electronic recoil data from the $\beta$ decays of ${}^3$H and ${}^{14}$C in the Large Underground Xenon (LUX) detector were used to tune the model, in addition to external data sets that allow for extrapolation beyond the LUX data-taking conditions. This paper also presents techniques used for modeling complicated temporal and spatial detector pathologies that can adversely affect data using a simplified model framework. The methods outlined in this report show an example of the robust applications possible with NEST v2.0, while also providing the final electronic recoil model and detector parameters that will used in the new analysis package, the LUX Legacy Analysis Monte Carlo Application (LLAMA), for accurate reproduction of the LUX data. As accurate background reproduction is crucial for the success of rare-event searches, such as dark matter direct detection experiments, the techniques outlined here can be used in other single-phase and dual-phase xenon detectors to assist with accurate ER background reproduction.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Finding simplicity: unsupervised discovery of features, patterns, and order parameters via shift-invariant variational autoencoders *

Abstract Recent advances in scanning tunneling and transmission electron microscopies (STM and STEM) have allowed routine generation of large volumes of imaging data containing information on the structure and functionality of materials. The experimental data sets contain signatures of long-range phenomena such as physical order parameter fields, polarization, and strain gradients in STEM, or standing electronic waves and carrier-mediated exchange interactions in STM, all superimposed onto scanning system distortions and gradual changes of contrast due to drift and/or mis-tilt effects. Correspondingly, while the human eye can readily identify certain patterns in the images such as lattice periodicities, repeating structural elements, or microstructures, their automatic extraction and classification are highly non-trivial and universal pathways to accomplish such analyses are absent. We pose that the most distinctive elements of the patterns observed in STM and (S)TEM images are similarity and (almost-) periodicity, behaviors stemming directly from the parsimony of elementary atomic structures, superimposed on the gradual changes reflective of order parameter distributions. However, the discovery of these elements via global Fourier methods is non-trivial due to variability and lack of ideal discrete translation symmetry. To address this problem, we explore the shift-invariant variational autoencoders (shift-VAEs) that allow disentangling characteristic repeating features in the images, their variations, and shifts that inevitably occur when randomly sampling the image space. Shift-VAEs balance the uncertainty in the position of the object of interest with the uncertainty in shape reconstruction. This approach is illustrated for model 1D data, and further extended to synthetic and experimental STM and STEM 2D data. We further introduce an approach for training shift-VAEs that allows finding the latent variables that comport to known physical behavior. In this specific case, the condition is that the latent variable maps should be smooth on the length scale of the atomic lattice (as expected for physical order parameters), but other conditions can be imposed. The opportunities and limitations of the shift VAE analysis for pattern discovery are elucidated.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Zinc isotope constraints on the cycling of carbon in the Bermuda mantle source

Volatile recycling and storage in the mantle transition zone (MTZ) is important for the refertilization of the upper mantle and is associated with the generation of high-µ (HIMU, where µ is 238 U/ 204 Pb) mantle. One way to probe the MTZ and the processes associated with mantle convection is to sample lavas that originate from the shallow mantle and were contaminated by upwelling from the MTZ, such as at the previously proposed shallow plume of Bermuda. Here we present the first δ 66 Zn isotopic compositions of Bermuda silica-undersaturated and silica-saturated lavas to explore the origin of the carbon-rich lithologies and the genesis of the large seamount found in the western North Atlantic Ocean. Contrasting with global δ 66 Zn data sets, our results (δ 66 Zn between 0.24 ± 0.04 and 0.41 ± 0.04) do not support direct sampling of recycled marine carbonates in the Bermuda HIMU mantle. Instead, we show that δ 66 Zn fractionation toward higher values is associated with magmatic processes and incorporation of carbon sourced from deep fluids associated with the formation of carbonatites. These carbon-rich fluids are likely sourced from the metasomatic reactions between the subducted cold slab of the Iapetus oceanic lithosphere ca. 500 Ma and the thickened continental lithospheric mantle of Pangea. Melting of this metasomatized mantle was triggered by the arrival of the Farallon slab to the eastern North American margin in the late Cenozoic via shallow convection.

Mazza, Sarah E. [Smith College, Northampton, MA (U↗
DOE OSTI2020

Assessing Historical Variability of South Asian Monsoon Lows and Depressions With an Optimized Tracking Algorithm

Abstract Cyclonic low‐pressure systems (LPS) produce abundant rainfall in South Asia, where they are traditionally categorized as monsoon lows, monsoon depressions, and more intense cyclonic storms. The India Meteorological Department (IMD) has tracked monsoon depressions for over a century, finding a large decline in their number in recent decades, but their methods have changed over time and do not include monsoon lows. This study presents a fast, objective algorithm for identifying monsoon LPS and uses it to assess interannual variability and trends in reanalyses. Variables and thresholds used in the algorithm are selected to best match a subjectively analyzed LPS data set while minimizing disagreement between four reanalyses in a training period. The stream function of 850 hPa horizontal wind is found to be optimal in this sense; it is less noisy than vorticity and represents the complete nondivergent wind, even when flow is not geostrophic. Using this algorithm, LPS statistics are computed for five reanalyses, and none show a detectable trend in monsoon depression counts since 1979. Both the Japanese 55‐year Reanalysis (JRA‐55) and the IMD data set show a step‐like reduction in depression counts when they began using geostationary satellite data, in 1979 and 1982, respectively; the 1958–2018 linear trend in JRA‐55, however, is smaller than in the IMD data set, and its error bar includes 0. There are more LPS in seasons with above‐average monsoon rainfall and in La Niña years, but few other large‐scale modes of interannual variability are found to modulate LPS counts, lifetimes, or track length consistently across reanalyses.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Microbial Taxa Distribution Data From 16S rRNA Analysis Of Desalination Operations At Carlsbad, CA And Tampa Bay, FL

This data set list the distribution of microbial taxa from three sets of sampling campaigns from unit operations in two large desalination facilities in the US conducted between March and May 2021. The desalination plants include the Claude "Bud" Lewis Carlsbad Desalination Plant in Carlsbad, California and the Seater Desalination facility in Tampa Bay, Florida.

16s rRNA gene analysis↗
DOE OSTI2022

Scaling SQL to the Supercomputer for Interactive Analysis of Simulation Data

AI and simulation workloads consume and generate large amounts of data that need to be searched, transformed and merged with other data. With the goal of treating data as a first-class citizen inside a traditionally compute-centric HPC environment, we explore how the use of accelerators and high-speed interconnects can speed up tasks which otherwise constitute bottlenecks in computational discovery workflows. BlazingSQL is SQL engine that runs natively on NVIDIA GPUs and supports internode communication for fast analytics on terabyte-scale tabular data sets. We show how a fast interconnect improves query performance if leveraged through the Unified Communication X (UCX) middleware. We envision that future computing platforms will integrate accelerated database query capabilities for immediate and interactive analysis of large simulation data.

Glaser, Jens↗
DOE OSTI2024

Efficient First-Order Algorithms for Large-Scale, Non-Smooth Maximum Entropy Models with Application to Wildfire Science

Maximum entropy (MaxEnt) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, MaxEnt models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for MaxEnt models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical MaxEnt models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models. Our proposed first-order algorithms leverage the Kullback–Leibler divergence to train large-scale and non-smooth MaxEnt models efficiently. For MaxEnt models with discrete probability distribution of n elements built from samples, each containing m features, the stepsize parameter estimation and iterations in our algorithms scale on the order of O(mn) operations and can be trivially parallelized. Moreover, the strong ℓ1 convexity of the Kullback–Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the art by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers.

Physics↗
DOE OSTI2021

Simulation of Hurricane Harvey flood event through coupled hydrologic-hydraulic models: Challenges and next steps

Using the 2017 Hurricane Harvey flood event as a test case, this study set up a series of sensitivity analyses to highlight three challenges associated with large-scale flood inundation modeling, including (a) model parameterization, (b) errors in digital elevation models, and (c) effects of reservoir retention. Driven by radar-based hourly rainfall data, a series of hydrologic-hydraulic models including the VIC hydrologic model, RAPID routing model, and Flood2D-GPU hydrodynamic model are set up over Harris County, Texas, to simulate flood inundation and hazards. The results demonstrate the importance of hydrologic parameters in improving flood modeling. For a large flood event such as Hurricane Harvey, the effect of the initial water depths is insignificant. The Manning's n values may increase the peak water depth by ~1%, the flood extents by 65km 2 , and the high danger zone by ~6%. On the contrary, the bathymetry correction factors may reduce the flood extent by ~1.4% and the high-danger zone by ~4%. Reducing the reservoir storage capacity to 1% may increase the flood extent by ~4% and the high-danger zone by ~17%. This study may provide supporting information to guide and prioritize the development of future high-performance computing hydrodynamic large-scale flood simulations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Measurements of the groomed jet radius and momentum splitting fraction with the soft drop and dynamical grooming algorithms in pp collisions at $ \sqrt{s}$ = 5.02 TeV

This article presents measurements of the groomed jet radius and momentum splitting fraction in pp collisions at $\sqrt{s}$ = 5.02 TeV with the ALICE detector at the Large Hadron Collider. Inclusive charged-particle jets are reconstructed at midrapidity using the anti-k T algorithm for transverse momentum 60 < p$_{T}^{ch jet}$ < 80 GeV/c. We report results using two different grooming algorithms: soft drop and, for the first time, dynamical grooming. For each grooming algorithm, a variety of grooming settings are used in order to explore the impact of collinear radiation on these jet substructure observables. These results are compared to perturbative calculations that include resummation of large logarithms at all orders in the strong coupling constant. We find good agreement of the theoretical predictions with the data for all grooming settings considered.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

An autonomous laboratory for the accelerated synthesis of novel materials

To close the gap between the rates of computational screening and experimental realization of novel materials, we introduce the A-Lab, an autonomous laboratory for the solid-state synthesis of inorganic powders. This platform uses computations, historical data from the literature, machine learning (ML) and active learning to plan and interpret the outcomes of experiments performed using robotics. Over 17 days of continuous operation, the A-Lab realized 41 novel compounds from a set of 58 targets including a variety of oxides and phosphates that were identified using large-scale ab initio phase-stability data from the Materials Project and Google DeepMind. Synthesis recipes were proposed by natural-language models trained on the literature and optimized using an active-learning approach grounded in thermodynamics. Analysis of the failed syntheses provides direct and actionable suggestions to improve current techniques for materials screening and synthesis design. The high success rate demonstrates the effectiveness of artificial-intelligence-driven platforms for autonomous materials discovery and motivates further integration of computations, historical knowledge and robotics.

36 MATERIALS SCIENCE↗