Handling noisy data in sparse model identification using subsampling and co-teaching
Not Available
Abdullah, Fahim↗
SEARCH · Engineering Papers
Results for “sparse data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Active learning a coarse-grained neural network model for bulk water from sparse training data
Active learning scheme to train neural network potentials for molecular simulations.
Loeffler, Troy D.↗
National population mapping from sparse survey data: A hierarchical Bayesian modeling framework to account for uncertainty
Population estimates are critical for government services, development projects, and public health campaigns. Such data are typically obtained through a national population and housing census. However, population estimates can quickly become inaccurate in localized areas, particularly where migration or displacement has occurred. Some conflict-affected and resource-poor countries have not conducted a census in over 10 y. We developed a hierarchical Bayesian model to estimate population numbers in small areas based on enumeration data from sample areas and nationwide information about administrative boundaries, building locations, settlement types, and other factors related to population density. We demonstrated this model by estimating population sizes in every 10- m grid cell in Nigeria with national coverage. These gridded population estimates and areal population totals derived from them are accompanied by estimates of uncertainty based on Bayesian posterior probabilities. The model had an overall error rate of 67 people per hectare (mean of absolute residuals) or 43% (using scaled residuals) for predictions in out-of-sample survey areas (approximately 3 ha each), with increased precision expected for aggregated population totals in larger areas. This statistical approach represents a significant step toward estimating populations at high resolution with national coverage in the absence of a complete and recent census, while also providing reliable estimates of uncertainty to support informed decision making.
99 GENERAL AND MISCELLANEOUS↗
Sparse thermal data for cellular automata modeling of grain structure in additive manufacturing
Grain growth in the wake of the melt pool formed during alloy-based additive manufacturing (AM) is complex and multifaceted, depending on parameters governing heat transport, fluid flow, and solidification itself. Cellular automata (CA) models have proven effective in providing computationally efficient and physically sound predictions of grain structure for several AM problems, but their efficiency is tied to the performance of heat transport models. CA models use only a small portion of the problem's temperature data (near the moving melt pool boundary), and much of the CA calculations do not affect the final result due to re-melting of material. Coupling of and communication between heat transport and solidification models, and eliminating operations irrelevant towards final grain structure prediction, will be necessary for using these methods for efficient simulation of large parts. In this work, we introduce a procedure of decoupling the CA from temperature field simulation, using files of relevant temperature data written by the heat transport model. This approach is validated against the standard coupling approach using data obtained through the computational fluid dynamics software OpenFOAM. Negligible differences are seen in grain size, volume, and texture distributions for multilayer simulation of test problems, while the quantity of temperature data for these test problems was reduced by four orders of magnitude (from 100s of GB to 10s of MB) and the code performance sped up by a factor of around 50. Furthermore, variability in microstructure as a function of cell size, substrate, time step, and nucleation parameters is studied, and it is found that cell sizes less than or equal to 1.67 μm and sufficiently small time steps yield Αstatistically equivalent microstructures. Finally, a potential use case for this CA approach—the layer-wise convergence in grain structure starting from extremes in initial grain size—is examined. This approach's ability to simulate expected trends in nucleation and epitaxial grain growth for large regions of microstructure, simulated independently of heat transport models themselves, should prove useful for investigation of various microstructure uncertainties and prediction of part-scale experimental results.
36 MATERIALS SCIENCE↗
Low Latency Detection of Sparse False Data Injections in Smart Grids
Not Available
24 POWER TRANSMISSION AND DISTRIBUTION↗
Rapid Analyses of Sparse Seismoacoustic Data Reveals the Timing and Size of the Accurate Energetic Systems Explosion
On 10 October 2025 an explosion occurred at a facility operated by Accurate Energetic Systems in Humphreys County, Tennessee. The incident resulted in 16 fatalities and created a debris field over several square kilometers. To address remaining questions about explosion timing and size, we collected about 20 seismic and 19 acoustic records of the blast from sensors up to hundreds of kilometers away. We then deployed 10 distinct physics-based, reduced order models (ROMs) that used validated geological structure and atmospheric conditions from the time of the event, along with observations of body- and surface-wave energy, as well as acoustic overpressure and phase duration. Each ROM predicted either timing, yield estimates, or both. We binned these estimates and their uncertainties according to each ROMs’ assumptions about confinement (aboveground, buried fully coupled, and buried partially coupled) and combined these estimates with other forensic data to conclude that the event occurred as a single, aboveground explosion on 10 December 2025 12:47:50.8 ±0.1 s with a yield equivalent to 11.8 [2.3,16.5] tons of Trinitrotoluene. Our estimates align with the Bureau of Alcohol, Tobacco, Firearms and Explosives inventory reports of 11–13 tons. This multimethod approach demonstrates the use of remotely observed geophysical data to rapidly aid conventional forensic investigations of accidental explosions.
45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
High Resolution Measurements and Modeling in the Arctic
This report details the activity of the project, "High Resolution Modeling and Measurements in the Arctic" spanning Fiscal Years 2016 - 2018 supported by the Sandia National Laboratories Laboratory Directed Research and Development (LDRD) program. The project's primary goal was to test the hypothesis that global climate model bias of low boundary layer clouds lacking liquid water in the Arctic could be improved by increasing horizontal resolution in the model. As model resolution is constrained by computational resources, four different model types were explored and compared to test the project's primary theory. Given the Arctic is a data-sparse region lacking robust data sets of liquid water in clouds, this project also obtained in situ measurements of low clouds with sensors on a tethered balloon system to constrain and compare with the models. Although other model biases remained, the liquid water path generally increased with resolution, supporting the original hypothesis.
54 ENVIRONMENTAL SCIENCES↗
Zero-Truncated Poisson Tensor Decomposition for Sparse Count Data
We propose a novel statistical inference paradigm for zero-inflated multiway count data that dispenses with the need to distinguish between true and false zero counts. Our approach ignores all zero entries and applies zero-truncated Poisson regression on the positive counts. Inference is accomplished via tensor completion that imposes low-rank structure on the Poisson parameter space. Our main result shows that an $\textit{N}$-way rank-R parametric tensor 𝓜 ϵ (0, ∞) $I$Χ∙∙∙Χ$I$ generating Poisson observations can be accurately estimated from approximately $IR^2 \text{log}^2_2(I)$ non-zero counts for a nonnegative canonical polyadic decomposition. Several numerical experiments are presented demonstrating that our zero-truncated paradigm is comparable to the ideal scenario where the locations of false zero counts are known $\textit{a priori}$.
97 MATHEMATICS AND COMPUTING↗
Leveraging Physics-based Surrogates for Efficient Density Estimation of Sparse Observable Data on Low-dimensional Manifolds.
Abstract not provided.
Yen, Tian Yu↗
Efficient Reconstruction of Resonant Cavity Fields from Sparse Sensor Data
Explore the source record for details and available documents.
Wallace, Jon William [Sandia National Laboratories↗
Deep Hyperspectral Classification for Data-Driven Sparse Sampling.
Abstract not provided.
Severa, William Mark↗
Bayesian Physics Informed Spatio-Temporal Network for Streamflow Data Imputation
Reliable reconstruction of incomplete streamflow records is critical for improving hydrological forecasting, flood preparedness, and water resource management. However, large observational gaps and uncertainties in governing physical parameters limit the accuracy of traditional statistical and machinelearning imputation frameworks. To address these challenges, we develop a Bayesian Physics-Informed Spatio-Temporal Network (BPI-STNet) that jointly captures spatial and temporal dependencies while enforcing hydrologic consistency through embedded physical constraints. The framework integrates a GraphSAGE-LSTM architecture to model spatial connectivity across gauges and temporal flow dynamics, coupled with a Bayesian update mechanism to estimate uncertain parameters in a simplified water-balance framework. Unlike conventional physics-informed networks that rely on sampling-based posterior estimation, BPI-STNet derives an analytic solution to the inverse problem, allowing closed-form Bayesian updates of uncertain parameters Λ={α,β,k} using Gaussian priors and likelihoods. Applied to daily observations from the Susquehanna River Basin (1980-2022), BPI-STNet achieves substantial improvements over a purely data-driven RGNN baseline, which reduced RMSE by 23 % and MAE by 9 %, and achieving an average NSE values up to 0.96. The results demonstrate that coupling Bayesian inference with physics-informed learning yields physically consistent, uncertainty-aware reconstructions that preserve the temporal persistence and statistical distribution of observed flows. The proposed framework establishes a generalizable paradigm for data-sparse hydrologic systems where both data fidelity and physical interpretability are essential.
Krishnan Kutty Ambika, Anukesh [ORNL] (ORCID:00000↗
Bayesian Approach to Estimation of Water Table Elevations Using Historical Rasters as Prior Information 2019 - 20430
In cases of complex but only partially known geology and a lack of spatial control in observation well locations, water table elevation estimation is very challenging. In some cases, auxiliary information, such as observations of the movement of tracers, operation of injection and extraction wells, and calibration of groundwater models against the historical elevation data, can be combined with expert judgement to estimate flow directions in areas of sparse data and to aid in the production of more reliable contour maps (and associated rasters) than could be produced by relying on sparse well elevation data alone. Given a historical sequence of these raster maps, the question arises how to automate, to the extent possible, the process of producing new raster maps to reflect data from previous times, the current data and the operation of expert judgement. One solution is to adopt a Bayesian point of view and to regard the historical well elevation data, auxiliary information and historical raster maps as prior information. The well elevations for water table wells, as well as those for injection/extraction wells and the data associated with other relevant variables, can be viewed as predictors for the raster surface. From this prior information, we can, conditional on the values of these predictors for a new time period, compute an expected value map and a standard deviation map for the new raster. These then can be taken to specify a prior predictive distribution for the pixels in the new raster map. Then we condition the pixels, corresponding to water level observation wells within the raster, on the observed values in those wells (which in general will differ from the regression estimate) for the new time period. Given the smoothness of the water table surface, we then smooth the surface of deviations from the mean surface, based on the variograms of the historical rasters, and add this smoothed surface to the regression mean surface. The error structure of the produced raster map is defined by the regression error structure and the error due to smoothing based on the estimated variograms. This methodology has been developed and is being further refined for groundwater monitoring and remediation at LANL. It is a very flexible method that can also be applied with a variety of other predictors applied to model the water level wells in the area of interest over the historical record. The smoothness of the spatial process and its possible evolution over time can then be estimated from the residuals from this regression. This can be augmented by expert hydrogeological opinion based on site topography and hydrogeology. (authors)
54 ENVIRONMENTAL SCIENCES↗
Multiscale Modeling Meets Machine Learning: What Can We Learn?
Machine learning is increasingly recognized as a promising technology in the biological, biomedical, and behavioral sciences. There can be no argument that this technique is incredibly successful in image recognition with immediate applications in diagnostics including electrophysiology, radiology, or pathology, where we have access to massive amounts of annotated data. However, machine learning often performs poorly in prognosis, especially when dealing with sparse data. This is a field where classical physics-based simulation seems to remain irreplaceable. In this review, we identify areas in the biomedical sciences where machine learning and multiscale modeling can mutually benefit from one another: Machine learning can integrate physics-based knowledge in the form of governing equations, boundary conditions, or constraints to manage ill-posted problems and robustly handle sparse and noisy data; multiscale modeling can integrate machine learn- ing to create surrogate models, identify system dynamics and parameters, analyze sensitivities, and quantify uncertainty to bridge the scales and understand the emergence of function. With a view towards applications in the life sciences, we discuss the state of the art of combining machine learning and multiscale modeling, identify applications and opportunities, raise open questions, and address potential challenges and limitations. We anticipate that it will stimulate discussion within the community of computational mechanics and reach out to other disciplines including mathematics, statistics, computer science, artificial intelligence, biomedicine, systems biology, and precision medicine to join forces towards creating robust and efficient models for biological systems.
machine learning, multiscale modeling, physics-bas↗
Operator learning for predicting multiscale bubble growth dynamics
We report simulating and predicting multiscale problems that couple multiple physics and dynamics across many orders of spatiotemporal scales is a great challenge that has not been investigated systematically by deep neural networks (DNNs). Herein, we develop a framework based on operator regression, the so-called deep operator network (DeepONet), with the long-term objective to simplify multiscale modeling by avoiding the fragile and time-consuming “hand-shaking” interface algorithms for stitching together heterogeneous descriptions of multiscale phenomena. To this end, as a first step, we investigate if a DeepONet can learn the dynamics of different scale regimes, one at the deterministic macroscale and the other at the stochastic microscale regime with inherent thermal fluctuations. Specifically, we test the effectiveness and accuracy of the DeepONet in predicting multirate bubble growth dynamics, which is described by a Rayleigh–Plesset (R–P) equation at the macroscale and modeled as a stochastic nucleation and cavitation process at the microscale by dissipative particle dynamics (DPD). First, we generate data using the R–P equation for multirate bubble growth dynamics caused by randomly time-varying liquid pressures drawn from Gaussian random fields (GRFs). Our results show that properly trained DeepONets can accurately predict the macroscale bubble growth dynamics and can outperform long short-term memory networks. We also demonstrate that the DeepONet can extrapolate accurately outside the input distribution using only very few new measurements. Subsequently, we train the DeepONet with DPD data corresponding to stochastic bubble growth dynamics. Although the DPD data are noisy and we only collect sparse data points on the trajectories, the trained DeepONet model is able to predict accurately the mean bubble dynamics for time-varying GRF pressures. Taken together, our findings demonstrate that DeepONets can be employed to unify the macroscale and microscale models of the multirate bubble growth problem, hence providing new insight into the role of operator regression via DNNs in tackling realistic multiscale problems and in simplifying modeling with heterogeneous descriptions.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
ECLEIRS: Exact conservation law embedded identification of reduced states for parameterized nonlinear conservation laws from sparse and noisy data
Multi-query applications such as parameter estimation, uncertainty quantification and design optimization for parameterized partial differential equation (PDE) systems are expensive. While reduced/latent state dynamics approaches for parameterized PDEs offer a viable alternative, these approaches rely on high-quality data and struggle with highly sparse spatiotemporal noisy measurements typically obtained from experiments. Furthermore, there is no guarantee that these models satisfy governing physical conservation laws. In this article, we propose a reduced state dynamics approach, referred to as ECLEIRS, that embeds exact conservation in the solution and flux representation by utilizing a space-time divergence-free neural network formulation. We compare ECLEIRS with other reduced state dynamics approaches, those that do not enforce any physical constraints and those with physics-informed loss functions, for three shock-propagation problems: 1-D advection, 1-D Burgers and 2-D Euler equations. In conclusion, the numerical experiments conducted in this study demonstrate that ECLEIRS provides the most accurate prediction of dynamics for unseen parameters even in the presence of highly sparse and noisy data.
97 MATHEMATICS AND COMPUTING↗
Quantum block encoding for one-pair semiseparable matrices
Quantum block encoding (QBE) is a crucial step in the development of most quantum algorithms, as it provides an embedding of a given matrix into a suitable larger unitary matrix. Historically, the development of efficient techniques for QBE has mostly focused on sparse matrices; less effort has been devoted to data-sparse (e.g., rank-structured) matrices. In this work we examine a particular case of rank structure, namely, one-pair semiseparable matrices. We present a new block encoding approach that relies on a suitable factorization of the given matrix as the product of triangular and diagonal factors. To encode the matrix, the algorithm needs $2\log(N)+7$ ancillary qubits. Assuming that the data input oracles can be implemented with polylogarithmic depth, or that a QRAM input model is available, our proposed method requires $\mathcal{O}({\rm polylog} (N))$ time and has an error of $\mathcal{O}(N^2)$, where $N$ is the matrix size.
Antonioli, Giacomo [Pisa U.; CERN] (ORCID:00090000↗