Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sparse Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Optical Variability of ICRF3 Quasars in the Pan-STARRS 3Pi Survey with Functional Principal Component Analysis

We make use of individual (epoch) detection data from the Pan-STARRS “3π” survey for 2863 optical ICRF3 counterparts in the five wavelength bands g, r, i, z, and y, published as part of the Data Release 2. A dedicated method based on the Functional Principal Component Analysis is developed for these sparse and irregularly sampled data. With certain regularization and normalization constraints, it allows us to obtain uniform and compatible estimates of the variability amplitudes and average magnitudes between the passbands and objects. We find that the starting assumption of affinity of the light curves for a given object at different wavelengths is violated for several percent of the sample. The distributions of rms variability amplitudes are strongly skewed toward small values, peaking at ∼0.1 mag with tails stretching to 2 mag. Statistically, the lowest variability is found for the r band and the largest for the reddest y band. A small “brighter-redder” effect is present, with amplitudes in y greater than amplitudes in g in 57% of the sample. The variability versus redshift dependence shows a strong decline with z toward redshift 3, which we interpret as the time dilation of the dominant time frequencies. The colors of radio-loud ICRF3 quasars are correlated with redshift in a complicated, wavy pattern governed by the emergence of brightest emission lines within the five passbands.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Use of Machine Learning on PMU Data for Transmission System Fault Analysis

Synchrophasor technology has been used for monitoring, control, and protection of bulk power system for over 10 years. Deployment of phasor measurement units (PMUs) in the USA power system has surpassed 3000 units installed in the transmission substations as stand-alone intelligent electronic devices (IEDs) or as a software add-on to other devices such as digital protective relays (DPRs) or digital fault recorders (DFRs). By now, thousands of terabytes of PMU data may have been captured and stored by various transmission system operators (TSOs) and independent system operators (ISOs). This creates an opportunity to deploy advanced machine learning (ML) techniques to detect and classify faults recorded by PMUs automatically to be used by the system operators for rapid, critical decision-making when manual analysis of the past or unfolding events is not feasible. In this paper we offer a brief background on how the automated fault analysis may be done using DPR and/or DFR data, and compare some of the legacy approaches to the new ML approaches in the context of the system-wide PMU recordings. We then offer insights from developing practical ML solutions that have been applied on field recordings captured by close to 450 PMUs from all three US interconnections (Western, Eastern and ERCOT) over two years (2016-2017). We identify and illustrate ML challenges we addressed: inaccurate data, data with scarce and temporally imprecise fault labels, data recorded by PMUs sparsely located at substations resulting in the fault records taken afar from the ends of the faulted lines, data containing only positive sequence values, and data taken at different voltage levels. We then illustrate the ML model results for fault analysis under different application scenarios. The novelty of this study is not only in the design, implementation, and performance analysis of the ML algorithms, but also in the use of advanced fault modelling and simulation approaches to improve the training results when developing supervised ML models for fault detection and classification. Extensive simulations of faults were conducted on a 14-bus power system to create a training dataset with over 1400 accurately labelled faults. This dataset was applied to enhance the accuracy of fault detection and classification of machine learning-based models trained with small number of labelled faults in large datasets recorded in the grid interconnections ranging from 5,000 to 70,000 buses.

Synchrophasors, Machine Learning, Fault Analysis, ↗
DOE OSTI2022

Characterizing and Mitigating Intraday Variability: Reconstructing Source Structure in Accreting Black Holes with mm-VLBI

The extraordinary physical resolution afforded by the Event Horizon Telescope has opened a window onto the astrophysical phenomena unfolding on horizon scales in two known black holes, M87* and Sgr A*. However, with this leap in resolution has come a new set of practical complications. Sgr A* exhibits intraday variability that violates the assumptions underlying Earth aperture synthesis, limiting traditional image reconstruction methods to short timescales and data sets with very sparse (u, v) coverage. We present a new set of tools to detect and mitigate this variability. We develop a data-driven, model-agnostic procedure to detect and characterize the spatial structure of intraday variability. This method is calibrated against a large set of mock data sets, producing an empirical estimator of the spatial power spectrum of the brightness fluctuations. We present a novel Bayesian noise modeling algorithm that simultaneously reconstructs an average image and statistical measure of the fluctuations about it using a parameterized form for the excess variance in the complex visibilities not otherwise explained by the statistical errors. These methods are validated using a variety of simulated data, including general relativistic magnetohydrodynamic simulations appropriate for Sgr A* and M87*. We find that the reconstructed source structure and variability are robust to changes in the underlying image model. We apply these methods to the 2017 EHT observations of M87*, finding evidence for variability across the EHT observing campaign. The variability mitigation strategies presented are widely applicable to very long baseline interferometry observations of variable sources generally, for which they provide a data-informed averaging procedure and natural characterization of inter-epoch image consistency.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Entropy Stable Conservative Flux Form Neural Networks

We propose an entropy-stable conservative flux form neural network (CFN) to predict the dynamics of unknown governing conservation laws. The design of the network is based on the entropy-stable, second-order, and non-oscillatory Kurganov-Tadmor (KT) scheme. The proposed entropy-stable CFN, hereafter referred to as ESCFN, uses slope limiting as a denoising mechanism, ensuring accurate predictions in both noisy and sparse observation environments, as well as in both smooth and discontinuous regions. Importantly, our method is designed to predict long term dynamics of the unknown conservation law exclusively from a short temporal window of observed data, that is, without oracle knowledge of the PDE or later-time solution profiles. Numerical experiments demonstrate that the ESCFN achieves both stability and conservation while maintaining accuracy over extended time domains, and successfully predicts shock propagation speeds in long-term simulations. Furthermore, it is also robust to both noisy and sparse data environments.

Hyperbolic conservation laws↗
DOE OSTI2021

MATAR: A performance portability and productivity implementation of data-oriented design with Kokkos

There is a need for simple, fast, and memory-efficient multidimensional data structures for dense and sparse storage that arise with numerical methods and in software applications. The data structures must perform equally well across multiple computer architectures, including CPUs and GPUs. For this purpose, we developed MATAR, a C++ software library that allows for simple creation and use of intricate data structures that is also portable across disparate architectures using Kokkos. Here, the performance aspect is achieved by forcing contiguous memory layout (or as close to contiguous as possible) for multidimensional and multi-size dense or sparse MATrix and ARray (hence, MATAR) types. Our results show that MATAR has the capability to improve memory utilization, performance, and programmer productivity in scientific computing. This is achieved by fitting more work into the available memory, minimizing memory loads required, and by loading memory in the most efficient order. This document describes the purpose of the work, the implementation of each of the data types, and the resulting performance both in some simple baseline test cases and in an application code.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

RARE EARTH AND CRITICAL ELEMENT RELATIONSHIPS TO COAL DEPOSIT LITHOLOGIES IN SOUTHWEST WYOMING AND NORTHWEST COLORADO

Rare earth elements (REE) and critical minerals (CM) are vital components in a wide variety of applications, not only in the renewable energy sector but also in military, telecommunication, and other civilian uses. The association of REE and CM with coal deposits is widely known but not thoroughly understood. While individual coal samples or beds themselves might not host significant concentrations of REE, recent studies indicate temporal relationships among some REE with coal beds and their immediately adjacent non-coal bounding strata, including interbeds. These strata most commonly include clays, carbonaceous shales, and ash beds, but other related strata are possible in different coal-bearing settings. Variations in REE and CM concentrations and their direct or indirect association with coal beds are not always predictable. This study presents recent findings from studying different associations between coal beds and adjacent rocks to identify possible relationships between REE-CM concentrations and the coals and adjacent rocks. For this study we collected and analyzed several dozen samples from the U.S. Geological Survey’s Core Research Center (CRC). Additionally, a small number of coal mine outcrops and their adjacent non-coal rocks were included. Most samples are of Cretaceous age. Previous work has shown a potential link between coals in the Upper Cretaceous Williams Fork Formation and REE occurrences in the Yampa ash bed in northwest Colorado. While data are uncommon and sparse, this possible link may suggest REE or CM enrichment from leaching or reworking of the ash bed. Trace element geochemistry, whole rock data, and microscopic examinations are used here to explore these relationships and guide future studies.

REE, CM, ash, volcanic ash, coal↗
DOE OSTI2023

Joint Inversion of Surface Electrical Resistivity Tomography and Seismic Refraction Data between the 200 Areas

Geologic stratigraphy on the Hanford Site influences groundwater and contaminant migration through the aquifer system and the vadose zone. The current geologic framework model (GFM) relies heavily on a sparse distribution of borehole data in some locations to map geologic contacts and hydrologic properties in the subsurface. Non-invasive geophysical methods such as electrical resistivity tomography (ERT), transient electromagnetic surveying, and seismic imaging are being used at Hanford to map subsurface structure in areas with limited well observations. This is to develop and mature the capability of geophysical methods to aid in GFM refinement, to identify regions of subsurface complexity, and for optimal well siting. A joint inversion of co-located seismic refraction and ERT data was carried out for data collected on a ~2.3-km profile between the 200 Areas on the Hanford Site. While ERT and seismic refraction images have sensitivity to overlapping physical properties (porosity, moisture content, lithology), the resolution and physics used to acquire each of these datasets are different and therefore information can be different or mutually complementary. Performing a joint inversion provides a reasonable option for a coherent, coupled interpretation for mutually complementary datasets. Between the 200 Areas, there are few boreholes to interpret the geologic framework model, and these data sets were obtained to provide a first line of evidence toward identifying stratigraphic structure. The seismic refraction and ERT data were independently inverted during fiscal year 2022 and broadly showed a two-layer structure with a trough-like feature that is ~1 km wide and upwards of 150 m deep. The depth of the trough feature was greater in the ERT image compared to the seismic image, which indicated a maximum depth of approximately 110 m. The objective of the joint inversion described in this report was to invert the seismic refraction and ERT data together while constraining the ERT image to be structurally similar to the seismic refraction image. The approach was applied using the geophysical inverse modeling program E4D, which has the capability to invert first-arrival times from seismic refraction data and ERT resistances using a “cross-gradient” constraint. The application of cross-gradient constraints with different weights produces ERT models that show a high degree of similarity within the upper 100 m (above ~120 m elevation). None of the ERT models show an improved structural similarity to the seismic result; therefore, it is recommended that further attempts to jointly interpret these models focus on petrophysics and image resolution. Petrophysical measurements of core samples would improve knowledge of what drives the ERT response in this region and, along with downhole geophysical measurements, could be used to “ground truth” the surface-based geophysical results. Image resolution studies would provide insight into which regions of the inverted images are reliable and which regions are poorly constrained.

58 GEOSCIENCES↗
DOE OSTI2022

Transfer learning driven design optimization for inertial confinement fusion

Transfer learning is a promising approach to create predictive models that incorporate simulation and experimental data into a common framework. In this technique, a neural network is first trained on a large database of simulations and then partially retrained on sparse sets of experimental data to adjust predictions to be more consistent with reality. Previously, this technique has been used to create predictive models of Omega [Humbird et al., IEEE Trans. Plasma Sci. 48, 61–70 (2019)] and NIF [Humbird et al., Phys. Plasmas 28, 042709 (2021); Kustowski et al., Mach. Learn. 3, 015035 (2022)] inertial confinement fusion (ICF) experiments that are more accurate than simulations alone. Here, in this work, we conduct a transfer learning driven hypothetical ICF campaign in which the goal is to maximize experimental neutron yield via Bayesian optimization. The transfer learning model achieves yields within 5% of the maximum achievable yield in a modest-sized design space in fewer than 20 experiments. Furthermore, we demonstrate that this method is more efficient at optimizing designs than traditional model calibration techniques commonly employed in ICF design. Such an approach to ICF design could enable robust optimization of experimental performance under uncertainty.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

A High Performance Sparse Tensor Algebra Compiler in MLIR

Sparse tensor algebra is widely used in many applications, including scientific computing, machine learning, and data analytics. The performance of sparse tensor algebra kernels strongly depends on the intrinsic characteristics of the input tensors, hence many storage formats are designed for tensors to achieve optimal performance for particular applications/architectures, which makes it challenging to implement and optimize every tensor operation of interest on a given architecture. We propose a tensor algebra domain-specific language (DSL) and compiler framework to automatically generate kernels for mixed sparse-dense tensor algebra operations. The proposed DSL provides high-level programming abstractions that resemble the familiar Einstein notation to represent tensor algebra operations. The compiler introduces a new Sparse Tensor Algebra dialect built on top of LLVM's extensible MLIR compiler infrastructure for efficient code generation while covering a wide range of tensor storage formats. Our compiler also leverages input-dependent code optimization to enhance data locality for better performance. Our results show that the performance of automatically generated kernels outperforms the state-of-the-art sparse tensor algebra compiler, with up to 20.92x, 6.39x, and 13.9x performance improvement over state-of-the-art tensor algebra compilers, for parallel SpMV, SpMM, and TTM, respectively.

Tian, Ruiqin↗
DOE OSTI2023

An Edge Alignment-Based Orientation Selection Method for Neutron Tomography

Neutron computed tomography (nCT) is a 3D char-acterization technique used to image the internal morphology or chemical composition of samples in biology and materials sciences. A typical workflow involves placing the sample in the path of a neutron beam, acquiring projection data at a predefined set of orientations, and processing the resulting data using an analytic reconstruction algorithm. Typical nCT scans require hours to days to complete and are then processed using conventional filtered back-projection (FBP), which performs poorly with sparse views or noisy data. Hence, the main methods in order to reduce overall acquisition time are the use of an improved sampling strategy combined with the use of advanced reconstruction methods such as model-based iterative reconstruction (MBIR). In this paper, we propose an adaptive orientation selection method in which an MBIR reconstruction on previously-acquired measurements is used to define an objective function on orientations that balances a data-fitting term promoting edge alignment and a regularization term promoting orientation diversity. Using simulated and experimental data, we demonstrate that our method produces high-quality reconstructions using significantly fewer total measurements than the conventional approach.

Yang, Diyu↗
DOE OSTI2025

High-throughput spin-bath characterization of spin defects in semiconductors

Detailed knowledge of the local environments of spin defects in semiconductors, such as nitrogenvacancy (NV) centers in diamond or divacancies in silicon carbide, is crucial for optimizing control and entanglement protocols in quantum sensing and information applications. However, at present a direct experimental characterization of individual defect environments is not scalable, as conventional spin-bath measurements are time consuming and difficult to automate. Achieving high-throughput characterization requires short experiments to probe the spin bath. However, with fewer and noisier measurements, the inverse problem of recovering spin-bath properties from measured data becomes ill posed, with multiple spin baths having a high likelihood of yielding the same data. In this work, we present a set of computational tools to resolve the ill-posed inverse problem of recovering the atomic positions and hyperfine couplings of random nuclei surrounding spin defects from sparse, noisy experimental coherence data, which can be obtained in hours. Here, we use a trans-dimensional Bayesian approach that incorporates ab initio data to yield full posterior distributions over nuclear spin environments, enabling robust recovery from limited data. We also provide practical tools and guidelines to determine the limits of detectability for hyperfine couplings under specific dynamical decoupling sequences and sampling conditions. In addition, we demonstrate how the tools developed here, in combination with ab initio simulations of spin baths, can guide the design of efficient experimental protocols for application-specific high-throughput screening. To showcase the utility of our approach, we apply it to design fast dynamical decoupling experiments to characterize the spin baths often individual NV centers in diamond. While the primary focus is on accelerating spin-bath characterization of spin defects, this Bayesian approach also lays the foundation for digital-twin studies of spin defects, where a virtual model of the spin-defect system evolves in real time with ongoing experimental measurements. Together, the set of tools we designed and applied paves the way for scalable deployment of spin defects in semiconductors for quantum sensing and information applications.

Bayesian methods↗