Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sparse representation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC

Abstract There is an emerging interest for tensor factorization applications in big‐data analytics and machine learning. To speed up the factorization of extra‐large datasets, organized in multidimensional arrays (also known as tensors), easy to compute compression‐based tensor representations, such as, Tucker and tensor train formats, are used to approximate the initial large‐tensor. Further, tensor factorization is used to extract latent features that can facilitate discoveries of new mechanisms and signatures hidden in the data, where the explainability of the latent features is of principal importance. Nonnegative tensor factorization extracts latent features that are naturally sparse and parts of the data, which makes them easily interpretable. However, to take into account available domain knowledge and subject matter expertise, often additional constraints need to be imposed, which lead us to canonical decomposition with linear constraints (CANDELINC), a canonical polyadic decomposition with rank deficient factors. In CANDELINC, Tucker compression is used as a preprocessing step, which lead to a larger residual error but to more explainable latent features. Here, we propose a nonnegative CANDELINC (nnCANDELINC) accomplished via a specific nonnegative Tucker decomposition; we refer to as minimal or canonical nonnegative Tucker. We derive several results required to understand the specificity of nnCANDELINC, focusing on the difficulties of preserving the nonnegative rank of a tensor to its Tucker core and comparing the real valued to nonnegative case. Finally, we demonstrate nnCANDELINC performance on synthetic and real‐world examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

SHMIP The subglacial hydrology model intercomparison Project

Subglacial hydrology plays a key role in many glaciological processes, including ice dynamics via the modulation of basal sliding. Owing to the lack of an overarching theory, however, a variety of model approximations exist to represent the subglacial drainage system. The Subglacial Hydrology Model Intercomparison Project (SHMIP) provides a set of synthetic experiments to compare existing and future models. We present the results from 13 participating models with a focus on effective pressure and discharge. For many applications (e.g. steady states and annual variations, low input scenarios) a simple model, such as an inefficient-system-only model, a flowline or lumped model, or a porous-layer model provides results comparable to those of more complex models. However, when studying short term (e.g. diurnal) variations of the water pressure, the use of a two-dimensional model incorporating physical representations of both efficient and inefficient drainage systems yields results that are significantly different from those of simpler models and should be preferentially applied. The results also emphasise the role of water storage in the response of water pressure to transient recharge. Finally, we find that the localisation of moulins has a limited impact except in regions of sparse moulin density.

58 GEOSCIENCES↗
DOE OSTI2026

Data-Driven Modeling and Correction of Vehicle Dynamics

We develop a data-driven framework for learning and correcting nonautonomous vehicle dynamics. Physics-based vehicle models are often simplified for tractability and therefore exhibit inherent model-form uncertainty, motivating the need for data-driven correction. Moreover, nonautonomous dynamics are governed by time-dependent control inputs, which pose challenges in learning predictive models directly from temporal snapshot data. To address these, we reformulate the vehicle dynamics via a local parameterization of the time-dependent inputs, yielding a modified system composed ofa sequence of local parametric dynamical systems. Here, we approximate these parametric systems using two complementary approaches. First, we employ the dimension reduction and interpolation in parameter space (DRIPS) methodology to construct efficient linear surrogate models, equipped with lifted observable spaces and manifold-based operator interpolation. This enables data-efficient learning of vehicle models whose dynamics admit accurate linear representations in the lifted spaces. Second, for more strongly nonlinear systems, we employ flow map learning (FML), a deep neural network (DNN) approach that approximates the parametric evolution map without requiring special treatment of nonlinearities. We further extend FML with a transfer-learning-based model correction procedure, enabling the correction of misspecified prior models using only a sparse set of high-fidelity or experimental measurements, without assuming a prescribed form for the correction term. Through a suite of numerical experiments on unicycle, simplified bicycle, and slip-based bicycle models, we demonstrate that DRIPS offers robust and highly data-efficient learning of nonautonomous vehicle dynamics, while FML provides expressive nonlinear modeling and effective correction of model-form errors under severe data scarcity.

data-driven modeling↗
DOE OSTI2024

Complexity of many-body interactions in transition metals via machine-learned force fields from the TM23 data set

Abstract This work examines challenges associated with the accuracy of machine-learned force fields (MLFFs) for bulk solid and liquid phases ofd-block elements. In exhaustive detail, we contrast the performance of force, energy, and stress predictions across the transition metals for two leading MLFF models: a kernel-based atomic cluster expansion method implemented using sparse Gaussian processes (FLARE), and an equivariant message-passing neural network (NequIP). Early transition metals present higher relative errors and are more difficult to learn relative to late platinum- and coinage-group elements, and this trend persists across model architectures. Trends in complexity of interatomic interactions for different metals are revealed via comparison of the performance of representations with different many-body order and angular resolution. Using arguments based on perturbation theory on the occupied and unoccupieddstates near the Fermi level, we determine that the large, sharpddensity of states both above and below the Fermi level in early transition metals leads to a more complex, harder-to-learn potential energy surface for these metals. Increasing the fictitious electronic temperature (smearing) modifies the angular sensitivity of forces and makes the early transition metal forces easier to learn. This work illustrates challenges in capturing intricate properties of metallic bonding with current leading MLFFs and provides a reference data set for transition metals, aimed at benchmarking the accuracy and improving the development of emerging machine-learned approximations.

Chemistry↗
DOE OSTI2025

Gaussian Process Regression under Computational and Epistemic Misspecification

Gaussian process regression is a classical kernel method for function estimation and data interpolation. In large data applications, computational costs can be reduced using low-rank or sparse approximations of the kernel. This paper investigates the effect of such kernel approximations on the interpolation error. We introduce a unified framework to analyze Gaussian process regression under important classes of computational misspecification: Karhunen-Loève expansions that result in low-rank kernel approximations, multiscale wavelet expansions that induce sparsity in the covariance matrix, and finite element representations that induce sparsity in the precision matrix. Furthermore, our theory also accounts for epistemic misspecification in the choice of kernel parameters.

Gaussian process regression↗
DOE OSTI2015

An exact formulation of the time-ordered exponential using path-sums

We present the path-sum formulation for the time-ordered exponential of a time-dependent matrix. The path-sum formulation gives the time-ordered exponential as a branched continued fraction of finite depth and breadth. The terms of the path-sum have an elementary interpretation as self-avoiding walks and self-avoiding polygons on a graph. Our result is based on a representation of the time-ordered exponential as the inverse of an operator, the mapping of this inverse to sums of walks on a graphs, and the algebraic structure of sets of walks. We give examples demonstrating our approach. We establish a super-exponential decay bound for the magnitude of the entries of the time-ordered exponential of sparse matrices. We give explicit results for matrices with commonly encountered sparse structures.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2018

Accuracy of Some Approximate Gaussian Filters for the Navier--Stokes Equation in the Presence of Model Error

Bayesian state estimation of a dynamical system from a stream of noisy measurements is important in many geophysical and engineering applications where high dimensionality of the state space, sparse observations, and model error pose key challenges. In this work, three computationally feasible, approximate Gaussian data assimilation/filtering algorithms are considered in various regimes of turbulent 2D Navier--Stokes dynamics in the presence of model error. The first source of error arises from the necessary use of reduced models for the forward dynamics of the filters, while a particular type of representation error arises from the finite resolution of observations which mix up information about resolved and unresolved dynamics. Two stochastically parameterized filtering algorithms, referred to as cSPEKF and GCF, are compared with 3DVAR---a prototypical time-sequential algorithm known to be accurate for filtering dissipative systems for a suitably inflated “background” covariance. We provide the first evidence that the stochastically parameterized algorithms, which do not rely on detailed knowledge of the underlying dynamics and do not require covariance inflation, can compete with or outperform an optimally tuned 3DVAR algorithm, and they can overcome competing sources of error in a range of dynamical scenarios.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Expanding the Domain of Applicability of Machine Learning Models with Limited Data for Drug Property Prediction

Accurate machine learning models for predicting small molecule interactions with biological targets are essential for therapeutic discovery, biothreat response, and computational drug design, but their performance is often limited for understudied targets with sparse experimental data. To address this challenge, we developed and evaluated methods to improve molecular property prediction under low-data conditions, using the NimA-related kinase (NEK) family as a proof-of-concept. This work focused on two complementary goals within the ATOM Modeling PipeLine (AMPL) and the Generative Molecular Design (GMD) loop: expanding model applicability through transfer learning, representation learning, feature scaling, sampling strategies, and active-learning-inspired compound selection; and enabling efficient virtual screening to prioritize compounds that balance predicted activity, design objectives, and synthetic accessibility.

organic↗
DOE OSTI2024

CO 2 storage site characterization using ensemble-based approaches with deep generative models

Estimating spatially distributed properties such as permeability from available sparse measurements is a great challenge in efficient subsurface CO 2 storage operations. In this paper, a deep generative model that can accurately capture complex subsurface structure is tested with an ensemble-based inversion method for accurate and accelerated characterization of CO 2 storage sites. We chose Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) for its realistic reservoir property representation and Ensemble Smoother with Multiple Data Assimilation (ES-MDA) for its robust data fitting and uncertainty quantification capability. WGAN-GP are trained to generate high-dimensional permeability fields from a low-dimensional latent space and ES-MDA then updates the latent variables by assimilating available measurements. Several subsurface site characterization examples including Gaussian, channelized, and fractured reservoirs are used to evaluate the accuracy and computational efficiency of the proposed method and the main features of the unknown permeability fields are characterized accurately with reliable uncertainty quantification. Furthermore, the estimation performance is compared with a widely-used variational, i.e., optimization-based, inversion approach, and the proposed approach outperforms the variational inversion method in several benchmark cases. We explain such superior performance by visualizing the objective function in the latent space: because of nonlinear and aggressive dimension reduction via generative modeling, the objective function surface becomes extremely complex while the ensemble approximation can smooth out the multi-modal surface during the minimization. This suggests that the ensemble-based approach works well over the variational approach when combined with deep generative models at the cost of forward model runs unless convergence-ensuring modifications are implemented in the variational inversion.

42 ENGINEERING↗
DOE OSTI2020

Conditional Karhunen-Loève expansion for uncertainty quantification and active learning in partial differential equation models

We use a conditional Karhunen-Lo` eve (KL) model to quantify and reduce uncertainty in a stochastic partial differential equation (SPDE) problem with partially-known space-dependent coefficient, Y (x). We assume that a small number of Y (x) measurements are available and model Y (x) with a KL expansion. We achieve reduction in uncertainty by conditioning the KL expansion coefficients on measurements. We consider two approaches for conditioning the KL expansion: In Approach 1, we condition the KL model first and then truncate it. In Approach 2, we first truncate the KL expansion and then condition it. We employ the conditional KL expansion together with Monte Carlo and sparse grid collocation methods to compute the moments of the solution of the SPDE problem. Uncertainty of the problem is further reduced by adaptively selecting additional observation locations using two active learning methods. Method 1 minimizes the variance of the PDE coefficient, while Method 2 minimizes the variance of the solution of the PDE. We demonstrate that conditioning leads to dimension reduction of the KL representation of Y (x). For a linear diffusion SPDE with uncertain log-normal coefficient, we show that Approach 1 provides a more accurate approximation of the conditional log-normal coefficient and solution of the SPDE than Approach 2 for the same number of random dimensions in a conditional KL expansion. Furthermore, Approach 2 provides a good estimate for the number of terms of the truncated KL expansion of the conditional field of Approach 1. Finally, we demonstrate that active learning based on Method 2 is more efficient for uncertainty reduction in the SPDE’s states (i.e., it leads to a larger reduction of the variance) than active learning using Method 2.

Conditioned Karhunen-Lo` eve expanion, machine lea↗
DOE OSTI2020

Simulating Observations of Southern Ocean Clouds and Implications for Climate

Southern Ocean (S. Ocean) clouds are important for climate prediction. Yet previous global climate models failed to accurately represent cloud phase distributions in this observation-sparse region. In this study, data from the Southern Ocean Clouds, Radiation, Aerosol, Transport Experimental Study (SOCRATES) experiment is compared to constrained simulations from a global climate model (the Community Atmosphere Model, CAM). Nudged versions of CAM are found to reproduce many of the features of detailed in situ observations, such as cloud location, cloud phase, and boundary layer structure. The simulation in CAM6 has improved its representation of S. Ocean clouds with adjustments to the ice nucleation and cloud microphysics schemes that permit more supercooled liquid. Comparisons between modeled and observed hydrometeor size distributions suggest that the modeled hydrometeor size distributions represent the dual peaked shape and form of observed distributions, which is remarkable given the scale difference between model and observations. Comparison to satellite observations of cloud physics is difficult due to model assumptions that do not match retrieval assumptions. Some biases in the model's representation of S. Ocean clouds and aerosols remain, but the detailed cloud physical parameterization provides a basis for process level improvement and direct comparisons to observations. This is crucial because cloud feedbacks and climate sensitivity are sensitive to the representation of S. Ocean clouds.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1989

Some methods of encoding simple visual images for use with a sparse distributed memory, with applications to character recognition

To study the problems of encoding visual images for use with a Sparse Distributed Memory (SDM), I consider a specific class of images- those that consist of several pieces, each of which is a line segment or an arc of a circle. This class includes line drawings of characters such as letters of the alphabet. I give a method of representing a segment of an arc by five numbers in a continuous way; that is, similar arcs have similar representations. I also give methods for encoding these numbers as bit strings in an approximately continuous way. The set of possible segments and arcs may be viewed as a five-dimensional manifold M, whose structure is like a Mobious strip. An image, considered to be an unordered set of segments and arcs, is therefore represented by a set of points in M - one for each piece. I then discuss the problem of constructing a preprocessor to find the segments and arcs in these images, although a preprocessor has not been developed. I also describe a possible extension of the representation.

Jaeckel, Louis A.↗
DOE OSTI2025

Bayesian learning with Gaussian processes for low-dimensional representations of time-dependent nonlinear systems

This work presents a data-driven method for learning low-dimensional time-dependent physics-based surrogate models whose predictions are endowed with uncertainty estimates. We use the operator inference approach to model reduction that poses the problem of learning low-dimensional model terms as a regression of state space data and corresponding time derivatives by minimizing the residual of reduced system equations. Standard operator inference models perform well with accurate training data that are dense in time, but producing stable and accurate models when the state data are noisy and/or sparse in time remains a challenge. Another challenge is the lack of uncertainty estimation for the predictions from the operator inference models. Our approach addresses these challenges by incorporating Gaussian process surrogates into the operator inference framework to (1) probabilistically describe uncertainties in the state predictions and (2) procure analytical time derivative estimates with quantified uncertainties. The formulation leads to a generalized least-squares regression and, ultimately, reduced-order models that are described probabilistically with a closed-form expression for the posterior distribution of the operators. The resulting probabilistic surrogate model propagates uncertainties from the observed state data to reduced-order predictions. Furthermore, we demonstrate the method is effective for constructing low-dimensional models of two nonlinear partial differential equations representing a compressible flow and a nonlinear diffusion–reaction process, as well as for estimating the parameters of a low-dimensional system of nonlinear ordinary differential equations representing compartmental models in epidemiology.

Data-driven model reduction↗
DOE OSTI2021

Seasonal Disorder in Urban Traffic Patterns: A Low Rank Analysis

This article proposes several advances to sparse nonnegative matrix factorization (SNMF) as a way to identify large-scale patterns in urban traffic data. The input to our model is traffic counts organized by time and location. Nonnegative matrix factorization additively decomposes this information, organized as a matrix, into a linear sum of temporal signatures. Penalty terms encourage this factorization to concentrate on only a few temporal signatures, with weights which are not too large. Our interest here is to quantify and compare the regularity of traffic behavior, particularly across different broad temporal windows. In addition to the rank and error, we adapt a measure introduced by Hoyer to quantify sparsity in the representation. Combining these, we construct several curves which quantify error as a function of rank (the number of possible signatures) and sparsity; as rank goes up and sparsity goes down, the approximation can be better and the error should decreases. Plots of several such curves corresponding to different time windows leads to a way to compare disorder/order at different time scalewindows. In this paper, we apply our algorithms and procedures to study a taxi traffic dataset from New York City. In this dataset, we find weekly periodicity in the signatures, which allows us an extra framework for identifying outliers as significant deviations from weekly medians. We then apply our seasonal disorder analysis to the New York City traffic data and seasonal (spring, summer, winter, fall) time windows. We do find seasonal differences in traffic order.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Bayesian differential programming for robust systems identification under uncertainty

This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling. This allows an efficient inference of the posterior distributions over plausible models with quantified uncertainty, while the use of sparsity-promoting priors enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed methods, including nonlinear oscillators, predator–prey systems and examples from systems biology. Taken together, our findings put forth a flexible and robust workflow for data-driven model discovery under uncertainty. All codes and data accompanying this article are available at https://bit.ly/34FOJMj .

Science & Technology - Other Topics↗
DOE OSTI2021

Scalable Techniques for Stochastic Power Flow Problems (Final Report)

The proposed research focuses on developing scalable algorithms for two-stage security-constrained OPF problems with AC power flow constraints, a class of problems complicated by (i) scale arising from a scenario representation; and (ii) the presence of nonlinearity, nonconvexity, and possibly second-stage discreteness or complementarity. Unfortunately, most existing solvers cannot contend with both challenges simultaneously; accordingly, the proposed research focuses on developing solution techniques that can both scale with the number of scenarios and contend with nonconvexity and second-stage complementarity. We consider three avenues for addressing such problems: (i) Variable sample-size SQP (VS-SQP) methods that combine sparse Quasi-Newton updates with a scalable variance-reduced stochastic gradient scheme for stochastic QP subproblems, allowing for contending with second-stage complementarity via regularization; (ii) Variable sample-size stochastic Interior-point (VS-sIP) schemes that propose a sampling-based regularized (to allow for contending with complementarity) interior-point schemes in which a Schur-complement technique is employed for decomposing the Newton direction computation step; (iii) Variable sample-size tractable ADMM (VS-tADMM) schemes combine variable sample-sizes with carefully designed techniques for resolving each of the nonconvex updates (by leveraging the QCQP structures). We intend to compare the three schemes using performance profiles in terms of solution quality, scalability, etc. and then select one scheme which will then be developed and further refined in Python for purposes of the GO competition.

42 ENGINEERING↗
DOE OSTI2018

Denoising Poisson phaseless measurements via orthogonal dictionary learning

Phaseless diffraction measurements recorded by CCD detectors are often affected by Poisson noise. In this paper, we propose a dictionary learning model by employing patches based sparsity in order to denoise such Poisson phaseless measurements. The model consists of three terms: (i) A representation term by an orthogonal dictionary, (ii) an L 0 pseudo norm of the coefficient matrix, and (iii) a Kullback-Leibler divergence term to fit phaseless Poisson data. Fast alternating minimization method (AMM) and proximal alternating linearized minimization (PALM) are adopted to solve the proposed model, and especially the theoretical guarantee of the convergence of PALM is provided. The subproblems for these two algorithms both have fast solvers, and indeed, the solutions for the sparse coding and dictionary updating both have closed forms due to the orthogonality of learned dictionaries. Numerical experiments for phase retrieval using coded diffraction and ptychographic patterns are conducted to show the efficiency and robustness of proposed methods, which, by preserving texture features, produce visually and quantitatively improved restored images compared with other phase retrieval algorithms without regularization and local sparsity promoting algorithms.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1990

The Cortex Transform as an image preprocessor for sparse distributed memory: An initial study

An experiment is described which was designed to evaluate the use of the Cortex Transform as an image processor for Sparse Distributed Memory (SDM). In the experiment, a set of images were injected with Gaussian noise, preprocessed with the Cortex Transform, and then encoded into bit patterns. The various spatial frequency bands of the Cortex Transform were encoded separately so that they could be evaluated based on their ability to properly cluster patterns belonging to the same class. The results of this study indicate that by simply encoding the low pass band of the Cortex Transform, a very suitable input representation for the SDM can be achieved.

Olshausen, Bruno↗