Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sparse Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Monthly Mean In Situ Surface Flux Observations Paired with Satellite-Derived and Reanalysis-Based Flux Data for the Great Lakes Region, 2001–2020

Surface radiative and turbulent heat fluxes over the Great Lakes strongly influence regional hydrological and meteorological processes, and their accurate representation is critical for numerical weather prediction and coupled atmosphere–lake modeling. However, direct flux observations are spatially sparse across the region, so gridded reanalysis and satellite-derived products are often used for climatological analyses and model evaluation despite differences in their flux representations. This dataset provides processed, quality-controlled, monthly mean surface flux observations from the Great Lakes Evaporation Network (GLEN), AmeriFlux, and the National Data Buoy Center, paired with spatiotemporally matched flux estimates from two reanalysis products, the fifth generation European Centre for Medium-Range Weather Forecasts (ECMWF) reanalysis dataset (ERA5) and the Modern Era Reanalysis for Research and Applications, version 2 (MERRA-2), and two satellite-derived products, the Clouds and Earth's Radiant Energy Systems Energy Balanced and Filled (CERES-EBAF) and the Cloud, Albedo and Surface Radiation dataset from AVHRR data - Edition 3 (CLARA-A3). The dataset includes sixteen observational stations with variable temporal coverage within 2001–2020. For each station, a CSV file contains monthly time series of available flux variables, including surface downwelling shortwave radiation (SW), surface downwelling longwave radiation (LW), sensible heat (SH) flux, and latent heat flux (LH), alongside matched gridded product values where available. Columns in the CSV file correspond to different variables sourced from each dataset, with column titles structured as "{dataset}_{variable}". Columns with relevant metadata are also provided in each CSV file, including station latitude and longitude, monthly timestamps, and the name of the sourced observational data. These files are structured for direct use in common analysis tools, including Microsoft Excel, Python pandas, and Python matplotlib. This dataset supports climatological analysis of the Great Lakes regional surface energy budget, evaluation of satellite-derived and reanalysis-based flux products, and development or validation of flux representations in numerical weather prediction and coupled atmosphere–lake models.

Great Lakes↗
DOE OSTI2025

Evaluation of Station Performance of the Idaho National Laboratory Seismic Monitoring Network Using Network Detection Thresholds

The Idaho National Laboratory (INL) Seismic Monitoring Network is located in eastern Idaho and monitors a portion of the intermountain seismic belt. It has been in place for 50 yr and has undergone several major changes, the most recent of which has been the transition to the Antelope real‐time acquisition system and the implementation of automatic phase picking algorithms to aid in analysis. This study discusses the efforts to evaluate the performance of the INL seismic monitoring network (and other surrounding stations) using the new real‐time acquisition system. The method outlined by Wilson et al. (2021) is used to develop an empirical relationship between the observability of local earthquakes as a function of magnitude and distance. This relationship is used to produce detection thresholds for Pwaves for all stations of interest. The INL seismic network has two main goals: monitor tectonic‐and volcanic‐related events and measure ground motions for input into seismic hazard analysis. Because of these two overall objectives, several seismic stations have been installed near critical facilities and, therefore, are not as quiet as stations that are used primarily for earthquake detection. This is reflected in their detection thresholds, which are much smaller for stations away from facilities. This study shows that the INL Seismic Monitoring Network is able to detect earthquakes near INL facilities with M L > 1.2, with redundancies built in to ensure this sensitivity even if data became unavailable from some stations. This study also shows “holes” in the monitoring network where the detection of smaller earthquakes is highly dependent on sparsely placed seismic stations. In conclusion, the results of this study will be used to govern plans for expansion of earthquake monitoring in Idaho and the surrounding region and to fine‐tune the detection thresholds for individual stations.

58 - GEOSCIENCES↗
DOE OSTI2020

Application of a long short-term memory for deconvoluting conductance contributions at charged ferroelectric domain walls

Ferroelectric domain walls are promising quasi-2D structures that can be leveraged for miniaturization of electronics components and new mechanisms to control electronic signals at the nanoscale. Despite the significant progress in experiment and theory, however, most investigations on ferroelectric domain walls are still on a fundamental level, and reliable characterization of emergent transport phenomena remains a challenging task. Here, we apply a neural-network-based approach to regularize local I ( V )-spectroscopy measurements and improve the information extraction, using data recorded at charged domain walls in hexagonal (Er 0.99 ,Zr 0.01 )MnO 3 as an instructive example. Using a sparse long short-term memory autoencoder, we disentangle competing conductivity signals both spatially and as a function of voltage, facilitating a less biased, unconstrained and more accurate analysis compared to a standard evaluation of conductance maps. The neural-network-based analysis allows us to isolate extrinsic signals that relate to the tip-sample contact and separating them from the intrinsic transport behavior associated with the ferroelectric domain walls in (Er 0.99 ,Zr 0.01 )MnO 3 . Our work expands machine-learning-assisted scanning probe microscopy studies into the realm of local conductance measurements, improving the extraction of physical conduction mechanisms and separation of interfering current signals.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Learning an Algebriac Multrigrid Interpolation Operator Using a Modified GraphNet Architecture

This work, building on previous efforts, develops a suite of new graph neural network machine learning architectures that generate data-driven prolongators for use in Algebraic Multigrid (AMG). Algebraic Multigrid is a powerful and common technique for solving large, sparse linear systems. Its effectiveness is problem dependent and heavily depends on the choice of the prolongation operator, which interpolates the coarse mesh results onto a finer mesh. Previous work has used recent developments in graph neural networks to learn a prolongation operator from a given coefficient matrix. In this paper, we expand on previous work by exploring architectural enhancements of graph neural networks. A new method for generating a training set is developed which more closely aligns to the test set. Asymptotic error reduction factors are compared on a test suite of 3-dimensional Poisson problems with varying degrees of element stretching. Results show modest improvements in asymptotic error factor over both commonly chosen baselines and learning methods from previous work.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Selective Dynamical Imaging of Interferometric Data

Recent developments in very long baseline interferometry (VLBI) have made it possible for the Event Horizon Telescope (EHT) to resolve the innermost accretion flows of the largest supermassive black holes on the sky. The sparse nature of the EHT’s (u, v)-coverage presents a challenge when attempting to resolve highly time-variable sources. We demonstrate that the changing (u, v)-coverage of the EHT can contain regions of time over the course of a single observation that facilitate dynamical imaging. These optimal time regions typically have projected baseline distributions that are approximately angularly isotropic and radially homogeneous. We derive a metric of coverage quality based on baseline isotropy and density that is capable of ranking array configurations by their ability to produce accurate dynamical reconstructions. We compare this metric to existing metrics in the literature and investigate their utility by performing dynamical reconstructions on synthetic data from simulated EHT observations of sources with simple orbital variability. We then use these results to make recommendations for imaging the 2017 EHT Sgr A* data set.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression

Here, we use Gaussian stochastic weight averaging (SWAG) to assess the epistemic uncertainty associated with neural-network-based function approximation relevant to fluid flows. SWAG approximates a posterior Gaussian distribution of each weight, given training data, and a constant learning rate. Having access to this distribution, it is able to create multiple models with various combinations of sampled weights, which can be used to obtain ensemble predictions. The average of such an ensemble can be regarded as the 'mean estimation', whereas its standard deviation can be used to construct 'confidence intervals', which enable us to perform uncertainty quantification (UQ) with regard to the training process of neural networks. We utilize representative neural-network-based function approximation tasks for the following cases: (i) a two-dimensional circular-cylinder wake; (ii) the DayMET dataset (maximum daily temperature in North America); (iii) a three-dimensional square-cylinder wake; and (iv) urban flow, to assess the generalizability of the present idea for a wide range of complex datasets. SWAG-based UQ can be applied regardless of the network architecture, and therefore, we demonstrate the applicability of the method for two types of neural networks: (i) global field reconstruction from sparse sensors by combining convolutional neural network (CNN) and multi-layer perceptron (MLP); and (ii) far-field state estimation from sectional data with two-dimensional CNN. We find that SWAG can obtain physically-interpretable confidence-interval estimates from the perspective of epistemic uncertainty. This capability supports its use for a wide range of problems in science and engineering.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

The 3D Lyman- α forest power spectrum from eBOSS DR16

We measure the three-dimensional power spectrum (P3D) of the transmitted flux in the Lyman-α (Ly α) forest using the complete extended Baryon Oscillation Spectroscopic Survey data release 16 (eBOSS DR16). This sample consists of ~205 000 quasar spectra in the redshift range 2 ≤ z ≤ 4 at an effective redshift z = 2.334. We propose a pair-count spectral estimator in configuration space, weighting each pair by exp( i k ∙ r), for wave vector k and pixel pair separation r, effectively measuring the anisotropic power spectrum without the need for fast Fourier transforms. This accounts for the window matrix in a tractable way, avoiding artefacts found in Fourier-transform based power spectrum estimators due to the sparse sampling transverse to the line of sight of Ly α skewers. We extensively test our pipeline on two sets of mocks: (i) idealized Gaussian random fields with a sparse sampling of Ly α skewers, and (ii) log-normal LyaCoLoRe mocks including realistic noise levels, the eBOSS survey geometry and contaminants. On eBOSS DR16 data, the Kaiser formula with a non-linear correction term obtained from hydrodynamic simulations yields a good fit to the power spectrum data in the range $(0.02 ≤ k ≤ 0.35)$ h Mpc -1 at the 1–2σ level with a covariance matrix derived from LyaCoLoRe mocks. We demonstrate a promising new approach for full-shape cosmological analyses of Ly α forest data from cosmological surveys such as eBOSS, the currently observing Dark Energy Spectroscopic Instrument and future surveys such as the Prime Focus Spectrograph, WEAVE-QSO, and 4MOST.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

An adaptive and stability-promoting layerwise training approach for sparse deep neural network architecture

This work presents a two-stage adaptive framework for progressively developing deep neural network (DNN) architectures that generalize well for a given training data set. In the first stage, a layerwise training approach is adopted where a new layer is added each time and trained independently by freezing parameters in the previous layers. We impose desirable structures on the DNN by employing manifold regularization, sparsity regularization, and physics-informed terms. We introduce a ε – δ – stability-promoting concept as a desirable property for a learning algorithm and show that employing manifold regularization yields a ε – δ stability-promoting algorithm. Further, we also derive the necessary conditions for the trainability of a newly added layer and investigate the training saturation problem. In the second stage of the algorithm (post-processing), a sequence of shallow networks is employed to extract information from the residual produced in the first stage, thereby improving the prediction accuracy. Numerical investigations on prototype regression and classification problems demonstrate that the proposed approach can outperform fully connected DNNs of the same size. Moreover, by equipping the physics-informed neural network (PINN) with the proposed adaptive architecture strategy to solve partial differential equations, we numerically show that adaptive PINNs not only are superior to standard PINNs but also produce interpretable hidden layers with provable stability. As a result, we also apply our architecture design strategy to solve inverse problems governed by elliptic partial differential equations.

42 ENGINEERING↗
DOE OSTI2025

Data-Driven Method for Groundwater-Level Mapping and Monitoring-Well Network Optimization at Hanford

This report summarizes the initial results and outcomes of a physics-informed, data-driven groundwater level (GWL) mapping capability for the Hanford Site. GWL mapping at Hanford is typically conducted annually and requires a significant amount of computational and expert resources, and it does not allow assessment of the informational value of specific monitoring wells. The proposed method produces spatially and temporally resolved fields consistent with sparse, irregularly sampled, and nonuniformly distributed well measurements. Implemented successfully, this capability will allow rapid mapping of groundwater levels and provide an opportunity to optimize monitoring activities (both location and sampling frequency) based on data information value evaluation. The approach integrates a diffusion-based generative model – trained on MODFLOW simulation data from the Plateau-to-River (P2R) model – with score-based data assimilation (SDA), allowing observation-conditioned mapping without retraining for each monitoring-network layout.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Ultrasparse View X-ray Computed Tomography for 4D Imaging

X-ray computed tomography (CT) is a noninvasive, nondestructive approach to imaging materials, material systems, and engineered components in two and three dimensions. Acquisition of three-dimensional (3D) images requires the collection of hundreds or thousands of through-thickness X-ray radiographic images from different angles. Such 3D data acquisition strategies commonly involve suboptimal temporal sampling for in situ and operando studies (4D imaging). Herein, we introduce a sparse-view imaging approach, Tomo-NeRF, which is capable of reconstructing high-fidelity 3D images from <10 twodimensional radiographic images. Experimental 2D and 3D X-ray images were used to test the reconstruction capability in two-view, four-view, and six-view scenarios. Tomo-NeRF is capable of reconstructing 3D images with a structural similarity of 0.9971–0.9975 and a voxel-wise accuracy of 81.83–89.59% from 2D experimentally obtained images. Furthermore, the reconstruction accuracy for the experimentally obtained images is less than the synthetic structures. Experimentally obtained images demonstrate a similarity of 0.9973–0.9984 and a voxelwise accuracy of 84.31–95.77%.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Pegasus V/Andromeda XXXIV–a newly discovered ultrafaint dwarf galaxy on the outskirts of Andromeda

ABSTRACT We report the discovery of an ultrafaint dwarf in the constellation of Pegasus. Pegasus V (Peg V)/Andromeda XXXIV was initially identified in the public imaging data release of the DESI Legacy Imaging Surveys and confirmed with deep imaging from Gemini/GMOS-N. The colour–magnitude diagram shows a sparse red giant branch (RGB) population and a strong overdensity of blue horizontal branch stars. We measure a distance to Peg V of $D=692^{+33}_{-31}$ kpc, making it a distant satellite of Andromeda with MV = −6.3 ± 0.2 and a half-light radius of rhalf = 89 ± 41 pc. It is located ∼260 kpc from Andromeda in the outskirts of its halo. The RGB is well fitted by a metal-poor isochrone with [Fe/H] = −3.2, suggesting it is very metal poor. This, combined with its blue horizontal branch, could imply that it is a reionization fossil. This is the first detection of an ultrafaint dwarf outside the deep Pan-Andromeda Archaeological Survey area, and points to a rich, faint satellite population in the outskirts of our nearest neighbour.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

A Comparison of Machine Learning Methods of Association Tested on Dense Nodal Arrays

The association of phase picks to form events is one of the fundamental components of seismology. Large and dense sensor networks, such as >1000 geophone arrays (and distributed acoustic sensing), offer unique challenges in association due to the vast numbers of observations and high likelihood of errant picks. In addition, the large number of stations can greatly increase the time it takes to perform the association. For this reason, machine learning (ML) methods might provide a more optimal method of association for such networks. In this work, we examine how well ML methods (e.g., Gaussian mixture model association, PhaseLink, and Graph Earthquake Neural Interpretation Engine) can incorporate dense seismic arrays into regional networks and how well they handle the increasing numbers of stations. Here, we test their capabilities on two dense seismic deployments, one within Rock Valley Nevada (52 nodes and a 9-station sparse local network), and the LArge-n Seismic Survey in Oklahoma dense nodal array (>1800 vertical-component geophones). Processing data from these two different styles of dense seismic deployments allows testing of how the ML algorithms can merge array data with a broader regional network, how they deal with poorly picked phases, and how they handle anthropogenic noise. We compare the ML-associated bulletins to those obtained using the Rapid Earthquake Association and Location algorithm, a more traditional method of association. We find that there are very small differences in results between the methods for small networks (<100 stations) with low pick rates. For large networks (>1000), there are enough errant picks that some of the ML methods start to create false events out of noise. We also find that the ML methods vary in computation time significantly but are all faster than the traditional method tested here.

58 GEOSCIENCES↗
DOE OSTI2024

Exploration with Scalable Gaussian Process Reinforcement Learning

Exploration is a challenging problem in reinforcement learning (RL), especially in environments with sparse rewards. Quantifying and utilizing the parametric uncertainty has been shown to be paramount for successful exploration [Osband et al., 2018]. Bayesian, or approximately Bayesian, methods present a principled means of estimating the parametric uncertainty in RL problems. Gaussian processes, nonparametric Bayesian models, are often impractical due to poor scalability and computational bottlenecks. We introduce a scalable Gaussian process RL (GPRL) method which directly induces sparsity in the covariance matrix to facilitate faster computation. This is a departure from previous GPRL methods which instead rely on data reduction and subsampling. We compare various covariance-based exploration techniques (Thompson sampling, upper confidence bound, and probabilistic maximum variance) which leverage our scalable GP framework in sparse reward environments. Finally, we show favorable comparison against the bootstrapped deep Q-Network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Sparse Matrix-Based HPC Tomography

Tomographic imaging has benefited from advances in X-ray sources, detectors and optics to enable novel observations in science, engineering and medicine. These advances have come with a dramatic increase of input data in the form of faster frame rates, larger fields of view or higher resolution, so high performance solutions are currently widely used for analysis. Tomographic instruments can vary significantly from one to another, including the hardware employed for reconstruction: from single CPU workstations to large scale hybrid CPU/GPU supercomputers. Furthermore, flexibility on the software interfaces and reconstruction engines are also highly valued to allow for easy development and prototyping. This paper presents a novel software framework for tomographic analysis that tackles all aforementioned requirements. The proposed solution capitalizes on the increased performance of sparse matrix-vector multiplication and exploits multi-CPU and GPU reconstruction over MPI. Furthermore, the solution is implemented in Python and relies on CuPy for fast GPU operators and CUDA kernel integration, and on SciPy for CPU sparse matrix computation. As opposed to previous tomography solutions that are tailor-made for specific use cases or hardware, the proposed software is designed to provide flexible, portable and high-performance operators that can be used for continuous integration at different production environments, but also for prototyping new experimental settings or for algorithmic development. The experimental results demonstrate how our implementation can even outperform state-of-the-art software packages used at advanced X-ray sources worldwide.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Technical note: Using long short-term memory models to fill data gaps in hydrological monitoring networks

Abstract. Quantifying the spatiotemporal dynamics in subsurface hydrological flows over a long time window usually employs a network of monitoring wells. However, such observations are often spatially sparse with potential temporal gaps due to poor quality or instrument failure. In this study, we explore the ability of recurrent neural networks to fill gaps in a spatially distributed time-series dataset. We use a well network that monitors the dynamic and heterogeneous hydrologic exchanges between the Columbia River and its adjacent groundwater aquifer at the U.S. Department of Energy's Hanford site. This 10-year-long dataset contains hourly temperature, specific conductance, and groundwater table elevation measurements from 42 wells with gaps of various lengths. We employ a long short-term memory (LSTM) model to capture the temporal variations in the observed system behaviors needed for gap filling. The performance of the LSTM-based gap-filling method was evaluated against a traditional autoregressive integrated moving average (ARIMA) method in terms of error statistics and accuracy in capturing the temporal patterns of river corridor wells with various dynamics signatures. Our study demonstrates that the ARIMA models yield better average error statistics, although they tend to have larger errors during time windows with abrupt changes or high-frequency (daily and subdaily) variations. The LSTM-based models excel in capturing both high-frequency and low-frequency (monthly and seasonal) dynamics. However, the inclusion of high-frequency fluctuations may also lead to overly dynamic predictions in time windows that lack such fluctuations. The LSTM can take advantage of the spatial information from neighboring wells to improve the gap-filling accuracy, especially for long gaps in system states that vary at subdaily scales. While LSTM models require substantial training data and have limited extrapolation power beyond the conditions represented in the training data, they afford great flexibility to account for the spatial correlations, temporal correlations, and nonlinearity in data without a priori assumptions. Thus, LSTMs provide effective alternatives to fill in data gaps in spatially distributed time-series observations characterized by multiple dominant frequencies of variability, which are essential for advancing our understanding of dynamic complex systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Exploiting voxel-sparsity for bone imaging with sparse-view cone-beam computed tomography

An optimization-based image reconstruction frame work is developed specifically for bone imaging. This framework exploits voxel-sparsity by use of ℓ 1 -norm image regularization and it enables image reconstruction from sparse-view cone-beam computed tomography (CBCT) acquisition. The effectiveness of the voxel-sparsity regularization is enhanced by using a blurred image representation. Ramp-filtering is included in the data discrepancy term and it has the effect of acting as a preconditioner, reducing the necessary number of iterations. The bone image reconstruction framework is demonstrated on CBCT data taken from an equine metacarpal condyle specimen.

Bone imaging↗
DOE OSTI2024

PreMevE‐MEO: Predicting Ultra‐Relativistic Electrons Using Observations From GPS Satellites

Abstract Ultra‐relativistic electrons with energies greater than or equal to two megaelectron‐volt (MeV) pose a major radiation threat to spaceborne electronics, and thus specifying those highly energetic electrons has a significant meaning to space weather communities. Here we report the latest progress in developing our predictive model for MeV electrons in the outer radiation belt. The new version, primarily driven by electron measurements made along medium‐Earth‐orbits (MEO), is called PREdictive MEV Electron (PreMevE)‐MEO model that nowcasts ultra‐relativistic electron flux distributions across the whole outer belt. Model inputs include >2 MeV electron fluxes observed in MEOs by a fleet of GPS satellites as well as electrons measured by one Los Alamos satellite in the geosynchronous orbit. We developed an innovative Sparse Multi‐Inputs Latent Ensemble NETwork (SmileNet) which combines convolutional neural networks with transformers, and we used long‐term in situ electron data from NASA's Van Allen Probes mission to train, validate, optimize, and test the model. It is shown that PreMevE‐MEO can provide hourly nowcasts with high model performance efficiency and high correlation with observations. This prototype PreMevE‐MEO model demonstrates the feasibility of making high‐fidelity predictions driven by observations from longstanding space infrastructure in MEO, thus has great potential of growing into an invaluable space weather operational warning tool.

79 ASTRONOMY AND ASTROPHYSICS↗