Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sparse data set”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Prong Segmentation using Point Set Transformers in Multiple View Neutrino Detectors

NOvA is a long-baseline neutrino experiment studying neutrino oscillations by detecting neutrinos from the NuMI beam at Fermilab. Its physics analysis relies on accurate prong segmentation, which involves matching each hit to its source particle and identifying the particle type. This task has commonly been addressed using a combination of traditional clustering algorithms and convolutional neural networks (CNNs). However, NOvA’s detector design presents data as two sparse and decoupled 2D images (XZ and YZ views) rather than a native 3D representation, posing a significant challenge for traditional CNN-based models. In this talk, we propose a novel neural network based on the Point Set Transformer. By treating detector hits as sparse point clouds and implementing a cross-view attention mechanism, our model enables efficient information mixing between both views. Evaluated on NOvA simulated data, our model achieves superior accuracy while requiring significantly fewer computational resources compared to other models. Furthermore, the model demonstrates great performance when applied to Liquid Argon Time Projection Chamber (LArTPC) data, which shows its potential as a universal prong segmentation algorithm for multiple view neutrino detectors.

Liu, Jiaxi [UC, Irvine]↗
NASA NTRS1997

The Influence of Correlated Crustal Signals in Modelling the Main Geomagnetic Field

Algorithms used in geomagnetic main-field modelling have for the most part treated the noise in the field measurements as if it were white. A major component of the noise consists of the field due to magnetization in the crust and it has been realized for some time that such signals are highly correlated at satellite altitude. Hence approximation by white noise, while of undoubted utility, is of unknown validity. In this paper we study two plausible statistical models for the crustal magnetization, in which the magnetization is a realization of a stationary, isotropic, random process. At a typical satellite altitude the associated fields exhibit significant correlation over ranges as great as 15 deg. or more, which introduces off-diagonal elements into the covariance matrix, elements that have usually been neglected in modelling procedures. Dealing with a full covariance matrix for a large data set would present a formidable computational challenge, but fortunately most of the entries in the covariance matrix are so small that they can be replaced by zeros. The resultant matrix comprises only about 3 per cent non-zero entries and thus we can take advantage of efficient sparse matrix techniques to solve the numerical system. We construct several main-field models based on vertical-component data from a selected 5 deg. by 5 deg. data set derived from the Magsat mission. Models with and without off-diagonal terms are compared.

Rygaard-Hjalsted, C.↗
DOE OSTI2020

A Satellite-Derived Upper-Ocean Stratification Data Set for the Tropical North Atlantic With Potential Applications for Hurricane Intensity Prediction

Upper-ocean stratification strongly impacts vertical mixing and the heat flux between the ocean and atmosphere, especially under extreme conditions of tropical cyclones (TCs). Knowledge of prestorm stratification is important for accurate TC intensity prediction. In situ observations of the tropical ocean have significantly increased in the past decade. However, they are still too sparse to resolve ocean stratification variability in near-real time and on small spatial scales. Here, in this study, based on long-term observations and an ocean reanalysis data set from 2004–2017, we investigate the possibility of retrieving upper-ocean stratification from sea surface temperature (SST), sea surface salinity (SSS), and sea surface height (SSH) using a simple regression method. It is found that more than 90% of the mean seasonal cycle and about 30% to 80% of temperature and salinity stratification anomalies can be reconstructed using surface data from either observations or an ocean reanalysis. Simple regression can be used with satellite observations to create a high-resolution, near-real-time-gridded ocean stratification data set that successfully reproduces both the large and mesoscale variability of ocean stratification. When used in a simple expression for TC-induced SST cooling, the satellite-derived stratification shows improvements over an ocean analysis in terms of variance explained of SST cooling, offering promise as a near-real-time indicator of the ocean's impact on TC intensification.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

High-throughput spin-bath characterization of spin defects in semiconductors

Detailed knowledge of the local environments of spin defects in semiconductors, such as nitrogenvacancy (NV) centers in diamond or divacancies in silicon carbide, is crucial for optimizing control and entanglement protocols in quantum sensing and information applications. However, at present a direct experimental characterization of individual defect environments is not scalable, as conventional spin-bath measurements are time consuming and difficult to automate. Achieving high-throughput characterization requires short experiments to probe the spin bath. However, with fewer and noisier measurements, the inverse problem of recovering spin-bath properties from measured data becomes ill posed, with multiple spin baths having a high likelihood of yielding the same data. In this work, we present a set of computational tools to resolve the ill-posed inverse problem of recovering the atomic positions and hyperfine couplings of random nuclei surrounding spin defects from sparse, noisy experimental coherence data, which can be obtained in hours. Here, we use a trans-dimensional Bayesian approach that incorporates ab initio data to yield full posterior distributions over nuclear spin environments, enabling robust recovery from limited data. We also provide practical tools and guidelines to determine the limits of detectability for hyperfine couplings under specific dynamical decoupling sequences and sampling conditions. In addition, we demonstrate how the tools developed here, in combination with ab initio simulations of spin baths, can guide the design of efficient experimental protocols for application-specific high-throughput screening. To showcase the utility of our approach, we apply it to design fast dynamical decoupling experiments to characterize the spin baths often individual NV centers in diamond. While the primary focus is on accelerating spin-bath characterization of spin defects, this Bayesian approach also lays the foundation for digital-twin studies of spin defects, where a virtual model of the spin-defect system evolves in real time with ongoing experimental measurements. Together, the set of tools we designed and applied paves the way for scalable deployment of spin defects in semiconductors for quantum sensing and information applications.

Bayesian methods↗
DOE OSTI2024

Community detection in hypergraphs via mutual information maximization

Abstract The hypergraph community detection problem seeks to identify groups of related vertices in hypergraph data. We propose an information-theoretic hypergraph community detection algorithm which compresses the observed data in terms of community labels and community-edge intersections. This algorithm can also be viewed as maximum-likelihood inference in a degree-corrected microcanonical stochastic blockmodel. We perform the compression/inference step via simulated annealing. Unlike several recent algorithms based on canonical models, our microcanonical algorithm does not require inference of statistical parameters such as vertex degrees or pairwise group connection rates. Through synthetic experiments, we find that our algorithm succeeds down to recently-conjectured thresholds for sparse random hypergraphs. We also find competitive performance in cluster recovery tasks on several hypergraph data sets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Growth and Stability of Stratiform Carrollite (CuCo2S4) in the Tenke-Fungurume Ore District, Central African Copperbelt

Abstract Carrollite (CuCo2S4) is the main ore mineral of the critical battery metal cobalt, yet, surprisingly, detailed characterization of its thermodynamic properties and of its trace element contents remain sparse in the scientific literature. To fill these knowledge gaps, we generated the first thermodynamic data set for carrollite using well-characterized samples obtained from the stratiform sulfide mineralization style at Fungurume 88 deposit in the Democratic Republic of Congo. Detailed mineralogical investigation of these same samples suggests that carrollite has limited ability to incorporate other ‘sweetener’ elements (except Cu and Ni) by substitution mechanisms. Modeling using CHNOSZ 2.0.0. software and an integrated Cu-Co-S-O thermodynamic data set provides insights into the stability of carrollite relative to other Co-bearing minerals under physicochemical conditions relevant to the Central African Copperbelt. Further exploration of these models indicates that dropping redox potential in the mineralizing system is insufficient to explain the mineral-scale and deposit-scale zonation patterns commonly observed in Cu-Co deposits of the Central African Copperbelt. Instead, factors such as increasing pH, decreasing T, decreasing sulfur activity, or some combination of changing physicochemical parameters may need to be invoked to explain these zonation trends. At the Fungurume 88 deposit, the absence of linnaeite (Co3S4) suggests that ore precipitation occurred at temperatures cooler than 211 °C. From our pH-logfO2 diagrams, and relative to higher temperatures, such a lower T regime is marked by a greater degree of offset between the modeled Cu and Co solubility contours, thus facilitating a more differentiated and zoned distribution of these two metals at the deposit scale. Our models are offered to the geoscience community as exceptionally useful tools for modeling and visualizing the Cu-Co-S-O systematics and may be applied to a variety of geological and geometallurgical questions in the Central African Copperbelt and other deposits elsewhere.

Mineralogy↗
NASA NTRS1991

Generation Of Surface Grids From Data Points

Computational procedure generates grids on complicated three-dimensional surfaces from sets of data points that lie on and specify those surfaces. Starting with grouping of possibly sparse surface points into lines and/or patches, procedure involves interpolation within and blending of lines and/or patches and possibly redistribution and reassembly of patches to obtain finished system of zonal patch grids that match at boundaries between them. Procedure semiautomated via computer program that performs all steps except selection of patches and interpolation points, left to discretion of user.

Luh, Raymond Ching-Chung↗
NASA NTRS1980

A hill-sliding strategy for initialization of Gaussian clusters in the multidimensional space

A hill sliding technique was devised to extract Gaussian clusters from the multivariate probability density estimate of sample data for the first step of iterative unsupervised classification. Each cluster was assumed to posses a unimodal normal distribution. A clustering function proposed distinguished elements of a cluster under formation from the rest in the feature space. Initial clusters were extracted one by one according to the hill sliding tactics. A dimensionless cluster compactness parameter was proposed as a universal measure of cluster goodness and used satisfactorily in test runs with LANDSAT multispectral scanner data. The normalized divergence, defined by the cluster divergence divided by the entropy of the entire sample data, was utilized as a general separability measure between clusters. An overall clustering objective function was set forth in terms of cluster covariance matrices, from which the cluster compactness measure could be deduced. Minimal improvement of initial data partitioning was evaluated by this objective function in eliminating scattered sparse data points. The hill sliding clustering technique developed herein has the potential applicability to decomposition any multivariate mixture distribution into a number of unimodal distributions when an appropriate distribution function to the data set is employed.

Park, J. K.↗
DOE OSTI2023

Machine learning-based inversion for acoustic impedance with large synthetic training data: Workflow and data characterization

Where wells are sparse or training data are difficult to label with high-quality wireline-derived impedance logs, machine learning (ML)-based inversion of acoustic impedance typically depends on small training data sets, leading to biased prediction. We have advanced a novel workflow that applies large synthetic seismic training data to reduce facies-related bias. Using a geologically realistic model as the truth model, we randomly select sparse seed wells to perform sequential Gaussian simulation (SGS) for impedance models of the same geometry and simulate facies variability. We implement random forest regression on 30 features extracted from the synthetic volume. We observe that more seed wells tend to reduce facies-induced bias by sampling more types of facies, resulting in a better prediction. We then focus on the responses of SGS models to facies changes, the number of seed wells necessary for a useful synthetic model, and how much a synthetic model can help ML-based inversion. Here, we observe that the SGS synthetic training model outperforms well-direct training in general. For modeled clastic shore-zone systems in Miocene Gulf of Mexico, two or more seed wells are necessary for a significant reduction of root-mean-square error and outliners, and improvement of facies imaging. In a field-data test, we apply a similar workflow to quantitatively predict acoustic impedance, which is then converted to a sand-volume map at a high-frequency sequence (10–100 m), revealing detailed facies and sandstone patterns. Such results are valuable in many geologic and engineering applications, such as hydrocarbon and CO 2 reservoir prospecting, reserve estimation, simulation, etc.

3D seismic↗
DOE OSTI2022

Numerical characterization of support recovery in sparse regression with correlated design

Sparse regression is employed in diverse scientific settings as a feature selection method. A pervasive aspect of scientific data is the presence of correlations between predictive features. These correlations hamper both feature selection and estimation and jeopardize conclusions drawn from estimated models. On the other hand, theoretical results on sparsity-inducing regularized regression have largely addressed conditions for selection consistency via asymptotics, and disregard the problem of model selection, whereby regularization parameters are chosen. In this numerical study, we address these issues through exhaustive characterization of the performance of several regression estimators, coupled with a range of model selection strategies. These estimators and selection criteria were examined across correlated regression problems with varying degrees of signal to noise, distributions of non-zero model coefficients, and model sparsity. Our results reveal a fundamental tradeoff between false positive and false negative control in all regression estimators and model selection criteria examined. Additionally, we numerically explore a transition point modulated by the signal-to-noise ratio and spectral properties of the design covariance matrix at which the selection accuracy of all considered algorithms degrades. Overall, we find that SCAD coupled with BIC or empirical Bayes model selection performs the best feature selection across the regression problems considered.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2017

Machine Learning Approaches to Increasing Value of Spaceflight Omics Databases

The number of spaceflight bioscience mission opportunities is too small to allow all relevant biological and environmental parameters to be experimentally identified. Simulated spaceflight experiments in ground-based facilities (GBFs), such as clinostats, are each suitable only for particular investigations -- a rotating-wall vessel may be 'simulated microgravity' for cell differentiation (hours), but not DNA repair (seconds) -- and introduce confounding stimuli, such as motor vibration and fluid shear effects. This uncertainty over which biological mechanisms respond to a given form of simulated space radiation or gravity, as well as its side effects, limits our ability to baseline spaceflight data and validate mission science. Machine learning techniques autonomously identify relevant and interdependent factors in a data set given the set of desired metrics to be evaluated: to automatically identify related studies, compare data from related studies, or determine linkages between types of data in the same study. System-of-systems (SoS) machine learning models have the ability to deal with both sparse and heterogeneous data, such as that provided by the small and diverse number of space biosciences flight missions; however, they require appropriate user-defined metrics for any given data set. Although machine learning in bioinformatics is rapidly expanding, the need to combine spaceflight/GBF mission parameters with omics data is unique. This work characterizes the basic requirements for implementing the SoS approach through the System Map (SM) technique, a composite of a dynamic Bayesian network and Gaussian mixture model, in real-world repositories such as the GeneLab Data System and Life Sciences Data Archive. The three primary steps are metadata management for experimental description using open-source ontologies, defining similarity and consistency metrics, and generating testing and validation data sets. Such approaches to spaceflight and GBF omics data may soon enable unique insight into which measured phenomena correlate to biological mechanisms that are truly affected by spaceflight conditions; which are most likely to be confounded by other variables; and which are insufficiently characterized, significantly increasing existing and future science return from ISS and spaceflight missions.

Gentry, Diana↗
NASA NTRSDate not supplied

Computational Fluid Dynamics Simulations of the Transonic Dynamics Tunnel Airstream Oscillator System

This paper presents a Computational Fluid Dynamics (CFD) model of the flow in the NASA Langley Research Center Transonic Dynamics Tunnel (TDT) with the Airstream Oscillator System (AOS) in operation. The TDT is a continuous-flow, closed circuit wind tunnel with a 16- by 16-foot slotted test section with cropped corners. The tunnel was originally built as the 19-ft Pressure Tunnel in 1938, but it was converted to the current transonic tunnel in the 1950s, with capabilities to use either air or heavy gas (R-134a) as the test medium. The TDT was also fitted with an AOS which can be used to create a gust field in the test section. To date, no computational analyses of the TDT involving the AOS have been performed. In this study, experimental data acquired of an oscillating airstream in the tunnel will be used to calibrate the computational analyses. A significant motivation of this work is to attempt to compare computational gust velocities to those recorded in the TDT literature. In addition to a validation of this model with experimental data, it may also be possible to supplement the experimental data with computational data. The experimental data is rather sparse and at only a few Mach numbers. Computational results may be able to expand the AOS data set. Another motivation for this work is that the Integrated Adaptive Wing Technology Maturation (IAWTM) semi-span model of the high-aspect-ratio CRM with 10 trailing edge control surfaces is currently being fabricated and will be delivered to the NASA Langley Transonic Dynamics Tunnel (TDT) for testing in late 2020. Among tests to be conducted, gust load alleviation (GLA) will be demonstrated at transonic conditions using the AOS.

Computational Fluid Dynamics↗
DOE OSTI2021

Discrete-Direct Model Calibration and Uncertainty Propagation Method Confirmed on Multi-Parameter Plasticity Model Calibrated to Sparse Random Field Data

A discrete direct (DD) model calibration and uncertainty propagation approach is explained and demonstrated on a 4-parameter Johnson-Cook (J-C) strain-rate dependent material strength model for an aluminum alloy. The methodology's performance is characterized in many trials involving four random realizations of strain-rate dependent material-test data curves per trial, drawn from a large synthetic population. The J-C model is calibrated to particular combinations of the data curves to obtain calibration parameter sets which are then propagated to “Can Crush” structural model predictions to produce samples of predicted response variability. These are processed with appropriate sparse-sample uncertainty quantification (UQ) methods to estimate various statistics of response with an appropriate level of conservatism. This is tested on 16 output quantities (von Mises stresses and equivalent plastic strains) and it is shown that important statistics of the true variabilities of the 16 quantities are bounded with a high success rate that is reasonably predictable and controllable. The DD approach has several advantages over other calibration-UQ approaches like Bayesian inference for capturing and utilizing the information obtained from typically small numbers of replicate experiments in model calibration situations—especially when sparse replicate functional data are involved like force–displacement curves from material tests. The DD methodology is straightforward and efficient for calibration and propagation problems involving aleatory and epistemic uncertainties in calibration experiments, models, and procedures.

42 ENGINEERING↗
NASA NTRS2015

Technical Report Series on Global Modeling and Data Assimilation: Soil Moisture Active Passive (SMAP) Project Assessment Report for the Beta-Release L4_SM Data Product - Volume 40

During the post-launch SMAP calibration and validation (Cal/Val) phase there are two objectives for each science data product team: 1) calibrate, verify, and improve the performance of the science algorithm, and 2) validate the accuracy of the science data product as specified in the science requirements and according to the Cal/Val schedule. This report provides an assessment of the SMAP Level 4 Surface and Root Zone Soil Moisture Passive (L4_SM) product specifically for the product's public beta release scheduled for 30 October 2015. The primary objective of the beta release is to allow users to familiarize themselves with the data product before the validated product becomes available. The beta release also allows users to conduct their own assessment of the data and to provide feedback to the L4_SM science data product team. The assessment of the L4_SM data product includes comparisons of SMAP L4_SM soil moisture estimates with in situ soil moisture observations from core validation sites and sparse networks. The assessment further includes a global evaluation of the internal diagnostics from the ensemble-based data assimilation system that is used to generate the L4_SM product. This evaluation focuses on the statistics of the observation-minus-forecast (O-F) residuals and the analysis increments. Together, the core validation site comparisons and the statistics of the assimilation diagnostics are considered primary validation methodologies for the L4_SM product. Comparisons against in situ measurements from regional-scale sparse networks are considered a secondary validation methodology because such in situ measurements are subject to upscaling errors from the point-scale to the grid cell scale of the data product. Based on the limited set of core validation sites, the assessment presented here meets the criteria established by the Committee on Earth Observing Satellites for Stage 1 validation and supports the beta release of the data. The validation against sparse network measurements and the evaluation of the assimilation diagnostics address Stage 2 validation criteria by expanding the assessment to regional and global scales.

ubRMSE↗
DOE OSTI2023

Ensemble Estimation of Historical Evapotranspiration for the Conterminous U.S.

Abstract Evapotranspiration (ET) is the largest component of the water budget, accounting for the majority of the water available from precipitation. ET is challenging to quantify because of the uncertainties associated with the many ET equations currently in use, and because observations of ET are uncertain and sparse. In this study, we combine information provided by available ET data and equations to produce a new monthly data set for ET for the conterminous U.S. (CONUS). These maps are produced from 1895 to 2018 at an 800 m spatial scale, marking a finer resolution than currently available products over this time period. In our approach, the relative performance of a suite of ET equations is assessed using water balance, flux tower, and remotely sensed ET estimates. At the observation locations, we use error distributions to quantify relative weights for the equations and use these in a modified Bayesian model averaging weighted ensemble approach. The relative weights are spatially generalized using a random forest regression, which is applied to wall‐to‐wall explanatory variable maps to generate CONUS‐wide relative weight maps and ensemble estimates. We assess the performance of the ensemble using a reserved subset of the observations and compare this performance against other national‐scale map products for historical to modern ET. The ensemble ET maps are shown to provide an improved accuracy over the alternative comparison products. These ET maps could be useful for a variety of hydrologic modeling and assessment applications that benefit from a long record, such as the study of periods of water scarcity through time.

Environmental Sciences & Ecology↗
NASA NTRS1994

Kelvin waves in total column ozone

Tropical Kelvin waves have been observed previously in ozone mixing ratio data from the SBUV (Solar Backscatter Ultraviolet) and LIMS (Limb Infrared Monitor of the Stratosphere) instruments on board the Nimbus-7 satellite. The present study investigates Kelvin wave features in total column ozone, using version 6 data from the Total Ozone Mapping Spectrometer (TOMS) instrument (also on Nimbus-7). Results show eastward-propagating zonal waves 1-2 with periods approx. 5-15 days, amplitudes approx. 3-5 Dobson Units (1-2% of the time mean), and latitudinal symmetry typical of Kelvin waves. The analyses and a linear model in this study suggest that the primary source of the perturbations is slow Kelvin waves in the lower-to-middle stratosphere. Maximum Kelvin wave signatures occur in conjunction with westward lower-to-middle stratospheric equatorial zonal winds (a quasi-biennial oscillation (QBO) wind modulation effect). The significance of these results is that the TOMS data are shown to be useful for investigations with global coverage of a major component of tropical stratospheric dynamics, Kelvin waves. The TOMS data set with its excellent coverage and high quality should be useful in validating model studies in the relatively data sparse and dynamically difficult tropical region.

Ziemke, J. R.↗
DOE OSTI2024

Multi-Level Structural Damage Characterization Using Sparse Acoustic Sensor Networks and Knowledge Transferred Deep Learning

Standard structural health monitoring techniques face well-known difficulties for comprehensive defect diagnosis in real-world structures that have structural, material, or geometric complexity. This motivates the exploration of machine-learning-based structural health monitoring methods in complex structures. However, creating sufficient training data sets with various defects is an ongoing challenge for data-driven machine (deep) learning algorithms. The ability to transfer the knowledge of a trained neural network from one component to another or to other sections of the same component would drastically reduce the required training data set. Also, it would facilitate computationally inexpensive machine learning based inspection systems. In this work, a machine-learning-based multi-level damage characterization is demonstrated with the ability to transfer trained knowledge within the sparse sensor network. A novel network spatial assistance and an adaptive convolution technique are proposed for efficient knowledge transfer within the deep learning algorithm. Proposed structural health monitoring method is experimentally evaluated on an aluminum plate with artificially induced defects. It was observed that the method improves the performance of knowledge transferred damage characterization by 50% during localization and 24% during severity assessment. Further, experiments using time windows with and without multiple edge reflections are studied. Results reveal that multiply scattered waves contain rich and deterministic defect signatures that can be mined using deep learning neural networks, improving the accuracy of both identification and quantification. In the case of a fixed sensor network, using multiply scattered waves shows 100% prediction accuracy at all levels of damage characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Heterogeneous Point Set Transformers for Segmentation of Multiple View Particle Detectors

NOvA is a long-baseline neutrino oscillation experiment that detects neutrino particles from the NuMI beam at Fermilab. Before data from this experiment can be used in analyses, raw hits in the detector must be matched to their source particles, and the type of each particle must be identified. This task has commonly been done using a mix of traditional clustering approaches and convolutional neural networks (CNNs). Due to the construction of the detector, the data is presented as two sparse 2D images: an XZ and a YZ view of the detector, rather than a 3D representation. We propose a point set neural network that operates on the sparse matrices with an operation that mixes information from both views. Our model uses less than 10% of the memory required using previous methods while achieving a 96.8% AUC score, a higher score than obtained when both views are processed independently (85.4%).

Robles, Edgar E. [UC, Irvine (main)]↗