Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sparse representations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Extreme sparsification of physics-augmented neural networks for interpretable model discovery in mechanics

Data-driven constitutive modeling with neural networks has received increased interest in recent years due to its ability to easily incorporate physical and mechanistic constraints and to overcome the challenging and time-consuming task of formulating phenomenological constitutive laws that can accurately capture the observed material response. However, even though neural network-based constitutive laws have been shown to generalize proficiently, the generated representations are not easily interpretable due to their high number of trainable parameters. Sparse regression approaches exist that allow for obtaining interpretable expressions, but the user is tasked with creating a library of model forms which by construction limits their expressiveness to the functional forms provided in the libraries. Here, in this work, we propose to train regularized physics-augmented neural network-based constitutive models utilizing a smoothed version of $L^0$-regularization. This aims to maintain the trustworthiness inherited by the physical constraints, but also enables interpretability which has not been possible thus far on any type of machine learning-based constitutive model where model forms were not assumed a priori but were actually discovered. During the training process, the network simultaneously fits the training data and penalizes the number of active parameters, while also ensuring constitutive constraints such as thermodynamic consistency. We show that the method can reliably obtain interpretable and trustworthy constitutive models for compressible and incompressible hyperelasticity, yield functions, and hardening models for elastoplasticity, using synthetic and experimental data. This work aims to set a new paradigm for interpretable machine learning models in the broad area of solid mechanics where low and limited data is available along with prior knowledge of physical constraints that the learned maps need to obey. This paradigm can potentially be extended to a broader spectrum of scientific exploration.

Data-driven constitutive models↗

Plant Root Characteristics and Dynamics in Arctic Tundra Ecosystems, 1960-2012

Plant roots play a critical role in ecosystem function in arctic tundra, but root dynamics in these ecosystems are poorly understood. To address this knowledge gap, we synthesized available literature on tundra roots, including their distribution, dynamics, and contribution to ecosystem carbon and nutrient fluxes, and we highlighted key aspects of their representation in terrestrial biosphere models. Across all tundra ecosystems, belowground plant biomass exceeded aboveground biomass, with the exception of polar desert tundra. Roots were shallowly-distributed in the thin layer of soil that thaws annually, and were often found in surface organic soil horizons. Root traits including distribution, chemistry, anatomy, and resource partitioning play an important role in controlling plant species competition, and therefore ecosystem carbon and nutrient fluxes, under changing climatic conditions, but have only been quantified for a small fraction of tundra plants. Further, the annual production and mortality of fine roots are key components of ecosystem processes in tundra, but extant data are sparse. Tundra root traits and dynamics should be the focus of future research efforts. Better representation of the dynamics and characteristics of tundra roots will improve the utility of models for evaluating the responses of tundra ecosystems to changing environmental conditions. Includes a data file in *.csv, and *.xlsx format and a *.pdf. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Paradeisos: A perfect hashing algorithm for many-body eigenvalue problems

Here, we describe an essentially perfect hashing algorithm for calculating the position of an element in an ordered list, appropriate for the construction and manipulation of many-body Hamiltonian, sparse matrices. Each element of the list corresponds to an integer value whose binary representation reflects the occupation of single-particle basis states for each element in the many-body Hilbert space. The algorithm replaces conventional methods, such as binary search, for locating the elements of the ordered list, eliminating the need to store the integer representation for each element, without increasing the computational complexity. Combined with the “checkerboard” decomposition of the Hamiltonian matrix for distribution over parallel computing environments, this leads to a substantial savings in aggregate memory. While the algorithm can be applied broadly to many-body, correlated problems, we demonstrate its utility in reducing total memory consumption for a series of fermionic single-band Hubbard model calculations on small clusters with progressively larger Hilbert space dimension.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A multiresolution adaptive wavelet method for nonlinear partial differential equations

We report the multiscale complexity of modern problems in computational science and engineering can prohibit the use of traditional numerical methods in multi-dimensional simulations. Therefore, novel algorithms are required in these situations to solve partial differential equations (PDEs) with features evolving on a wide range of spatial and temporal scales. To meet these challenges, we present a multiresolution wavelet algorithm to solve PDEs with significant data compression and explicit error control. We discretize in space by projecting fields and spatial derivative operators onto wavelet basis functions. We provide error estimates for the wavelet representation of fields and their derivatives. Then, our estimates are used to construct a sparse multiresolution discretization which guarantees the prescribed accuracy. Additionally, we embed a predictor-corrector procedure within the temporal integration to dynamically adapt the computational grid and maintain the accuracy of the solution of the PDE as it evolves. We present examples to highlight the accuracy and adaptivity of our approach.

97 MATHEMATICS AND COMPUTING↗

Object detection approach using generative sparse, hierarchical networks with top-down and lateral connections for combining texture/color detection and shape/contour detection

An approach to detecting objects in an image dataset may combine texture/color detection, shape/contour detection, and/or motion detection using sparse, generative, hierarchical models with lateral and top-down connections. A first independent representation of objects in an image dataset may be produced using a color/texture detection algorithm. A second independent representation of objects in the image dataset may be produced using a shape/contour detection algorithm. A third independent representation of objects in the image dataset may be produced using a motion detection algorithm. The first, second, and third independent representations may then be combined into a single coherent output using a combinatorial algorithm.

97 MATHEMATICS AND COMPUTING↗

Object detection approach using generative sparse, hierarchical networks with top-down and lateral connections for combining texture/color detection and shape/contour detection

An approach to detecting objects in an image dataset may combine texture/color detection, shape/contour detection, and/or motion detection using sparse, generative, hierarchical models with lateral and top-down connections. A first independent representation of objects in an image dataset may be produced using a color/texture detection algorithm. A second independent representation of objects in the image dataset may be produced using a shape/contour detection algorithm. A third independent representation of objects in the image dataset may be produced using a motion detection algorithm. The first, second, and third independent representations may then be combined into a single coherent output using a combinatorial algorithm.

97 MATHEMATICS AND COMPUTING↗

Advanced modeling of $\mathrm{DER}$s and $\mathrm{UPFC}$ devices with Sparse Tableau Formulation for $\mathrm{ACOPF}$

Distributed energy resources (DERs) are projected to be significant contributors in electric power grid modernization efforts. Yet many distribution and transmission system operators are concerned about potential reliability and security impacts of high penetration of DERs, and cite the need of greater visibility and controllability of decision variables associated with these DERs. Tractable modeling of DERs in AC optimal power flow (ACOPF) is one of the barriers to their integration into the existing operation, planning and market paradigm. Similarly, as more intermittent renewable resources are deployed, power flow control becomes more valuable, and accurate representation of Flexible AC Transmission System (FACTS) devices in ACOPF will likely grow in importance. Here this paper focuses on the detailed representation of substations with the high penetration of DERs, and on the Unified Power Flow Controller (UPFC) FACTS devices, in OPF. To this end, a novel Sparse Tableau Formulation (STF) for ACOPF is used to accurately model such features as substation reconfiguration to accommodate DERs, and to represent decision variables appropriate to the voltage source converters within UPFCs. Numerical studies are provided to demonstrate the impacts of detailed DER and UPFC representations on optimal operation, and the computational efficiency.

42 ENGINEERING↗

Efficient Sparse Matrix-Matrix Multiplication on Heterogeneous High Performance Systems

The efficient implementation of sparse matrix-matrix multiplications on high performance parallel machines poses several challenges: large size of input matrices, compressed representation, density of the output matrices, partitioning and load balancing of matrices that present parts with large differences in density and, thus, in computation times. In this paper we show how, starting from the requirements of such application, we developed a framework that allows its efficient implementation on heterogeneous clusters. We introduce a task based programming model and a runtime supported execution model which provides dynamic load balancing on clusters composed by CPUs and GPUs, allowing better utilization of the system while easing the handling of sparse matrices. Our results show that our solution, which co-designs the application together with the programming model and the runtime system, is able to obtain significant speed-ups due to a virtually perfect load balancing with respect to other programming approaches.

97 MATHEMATICS AND COMPUTING↗

Graph theory inspired anomaly detection at the LHC

Designing model-independent anomaly detection algorithms for analyzing LHC data remains a central challenge in the search for new physics, due to the high dimensionality of collider events. In this work, we develop a graph autoencoder as an unsupervised, model-agnostic tool for anomaly detection, using the LHC Olympics dataset as a benchmark. By representing jet constituents as a graph, we introduce a method to systematically control the information available to the model through sparse graph constructions that serve as physically motivated inductive biases. Specifically, (1) we construct graph autoencoders based on locally rigid Laman graphs and globally rigid unique graphs, and (2) we explore the clustering of jet constituents into subjets to interpolate between high- and low-level input representations. We obtain the best performance, measured in terms of the Significance Improvement Characteristic curve for an intermediate level of subjet clustering and certain sparse unique graph constructions. We further investigate the role of graph connectivity in jet classification tasks. Our results demonstrate the potential of leveraging graph-theoretic insights to refine and increase the interpretability of machine learning tools for collider experiments.

Automation↗

Results of the second Ice Shelf–Ocean Model Intercomparison Project (ISOMIP+)

Ocean-driven basal melting of Antarctic ice shelves plays an important role in the mass loss of the Antarctic Ice Sheet. Ice shelf cavity-resolving ocean models are a valuable tool for understanding ice shelf-ocean interactions and for simulating projections of ice shelf and ocean states under future climate. Designed to assess the current state of ice shelf–ocean modelling, the second Ice Shelf–Ocean Model Intercomparison Project, ISOMIP+, consists of 12 ocean model configurations submitted with a common, idealised experimental setup. Here, we focus on the experiments Ocean0–2 (Asay-Davis et al., 2016), which are ocean models with idealised, static ice shelf geometries, but where the ocean reaches a balance with prescribed far-field ocean conditions. Different thermal transfer coefficient values (ranging from 0.011 to 0.2) are used for each model in the melting parameterisation to achieve a common, tuned melt rate since the models cover a range of types of vertical coordinates, ice–ocean boundary layer treatments, and numerical schemes. These model differences lead to spread in the resultant ocean properties, circulation, boundary-layer structure and spatial distribution of melting. We also highlight similarities between models, such as a shared linear relationship across most models between melt rate and overturning and barotropic streamfunctions during the spin-up and spin-down, demonstrating a robust relationship between melt and circulation across models and forcing conditions. The ISOMIP+ results provide a systematic comparison of ice shelf cavity-capable ocean models. However, we also demonstrate the need for realistic ice shelf–ocean model intercomparison projects (some already underway) to assess model biases and inter-model variation against sparse observations. Further research is needed to understand the differences between models and further improve our modelled representations of the ice–ocean boundary layer and ice shelf cavity circulation.

54 ENVIRONMENTAL SCIENCES↗

ReLU, Sparseness, and the Encoding of Optic Flow in Neural Networks

Accurate self-motion estimation is critical for various navigational tasks in mobile robotics. Optic flow provides a means to estimate self-motion using a camera sensor and is particularly valuable in GPS- and radio-denied environments. The present study investigates the influence of different activation functions—ReLU, leaky ReLU, GELU, and Mish—on the accuracy, robustness, and encoding properties of convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) trained to estimate self-motion from optic flow. Our results demonstrate that networks with ReLU and leaky ReLU activation functions not only achieved superior accuracy in self-motion estimation from novel optic flow patterns but also exhibited greater robustness under challenging conditions. The advantages offered by ReLU and leaky ReLU may stem from their ability to induce sparser representations than GELU and Mish do. Our work characterizes the encoding of optic flow in neural networks and highlights how the sparseness induced by ReLU may enhance robust and accurate self-motion estimation from optic flow.

97 MATHEMATICS AND COMPUTING↗

Energy Efficient Stochastic-Based Deep Spiking Neural Networks for Sparse Datasets

With large deep neural networks (DNNs) neces- sary to solve complex and data-intensive problems, energy efficiency is a key bottleneck for effectively deploying DL in the real world. Deep spiking NNs have gained much research at- tention recently due to the interest in building biological neural networks and the availability of neuromorphic platforms, which can be orders of magnitude more energy efficient compared to CPUs and GPUs. Although spiking NNs have proven to be an efficient technique for solving many machine learning and computer vision problems, to the best of our knowledge, this is the first attempt to adapt spiking NNs to sparse datasets. In this paper, we study the behaviour of spiking NNs in handling NLP datasets and the sparsity in their data representation. Then, we propose a novel framework for spiking NN using the concept of stochastic computing. Specifically, instead of generating spike trains with firing rates proportional to the intensity of each value in the feature set separately, the whole feature set is treated as a distribution function and a stochastic spiking train that follow this distribution is generated. This framework reduces the connectivity between NN layers from O(N) to O(logN). Also, it encodes input data differently and make suitable to handle sparse datasets. Finally, the framework achieves high energy efficiency since it uses Integrate and Fire neurons same as conventional spiking NNs. The results show that our proposed stochastic-based SNN achieves nearly the same accuracy as the original DNN on MNIST dataset, and it has better performance than state- of-the-art SNN. Besides that stochastic-based SNN is energy efficient, where the fully connected DNN, the conventional SNN, and the data normalized SNN consume 38.24, 1.83, and 1.85 -times more energy than the stochastic-based SNN, respectively. For sparse datasets, including IMDb and In-House clinical datasets, stochastic-based SNN achieves performance comparable to that of the conventional DNN. However, the conventional spiking NN has a significant decline in classification accuracy.

Alawad, Mohammed↗

Graph Partitioning and Sparse Matrix Ordering using Reinforcement Learning and Graph Neural Networks

We present a novel method for graph partitioning, based on reinforcement learning and graph convolutional neural networks. Our approach is to recursively partition coarser representations of a given graph. The neural network is implemented using SAGE graph convolution layers, and trained using an advantage actor critic (A2C) agent. We present two variants, one for finding an edge separator that minimizes the normalized cut or quotient cut, and one that finds a small vertex separator. The vertex separators are then used to construct a nested dissection ordering to permute a sparse matrix so that its triangular factorization will incur less fill-in. The partitioning quality is compared with partitions obtained using METIS and SCOTCH, and the nested dissection ordering is evaluated in the sparse solver SuperLU. Our results show that the proposed method achieves similar partitioning quality as METIS and SCOTCH. Furthermore, the method generalizes across different classes of graphs, and works well on a variety of graphs from the SuiteSparse sparse matrix collection.

97 MATHEMATICS AND COMPUTING↗

A novel software and conceptual design of the hardware platform for intensity modulated radiation therapy

Purpose: To deliver high quality intensity modulated radiotherapy (IMRT) using a novel generalized sparse orthogonal collimators (SOCs), the authors introduce a novel direct aperture optimization (DAO) approach based on discrete rectangular representation. Methods: A total of seven patients—two glioblastoma multiforme, three head & neck (including one with three prescription doses), and two lung—were included. 20 noncoplanar beams were selected using a column generation and pricing optimization method. The SOC is a generalized conventional orthogonal collimators with N leaves in each collimator bank, where N = 1, 2, or 4. SOC degenerates to conventional jaws when N = 1. For SOC-based IMRT, rectangular aperture optimization (RAO) was performed to optimize the fluence maps using rectangular representation, producing fluence maps that can be directly converted into a set of deliverable rectangular apertures. In order to optimize the dose distribution and minimize the number of apertures used, the overall objective was formulated to incorporate an L2 penalty reflecting the difference between the prescription and the projected doses, and an L1 sparsity regularization term to encourage a low number of nonzero rectangular basis coefficients. The optimization problem was solved using the Chambolle–Pock algorithm, a first-order primal–dual algorithm. Performance of RAO was compared to conventional two-step IMRT optimization including fluence map optimization and direct stratification for multileaf collimator (MLC) segmentation (DMS) using the same number of segments. For the RAO plans, segment travel time for SOC delivery was evaluated for the N = 1, N = 2, and N = 4 SOC designs to characterize the improvement in delivery efficiency as a function of N. Results: Comparable PTV dose homogeneity and coverage were observed between the RAO and the DMS plans. The RAO plans were slightly superior to the DMS plans in sparing critical structures. On average, the maximum and mean critical organ doses were reduced by 1.94% and 1.44% of the prescription dose. The average number of delivery segments was 12.68 segments per beam for both the RAO and DMS plans. The N = 2 and N = 4 SOC designs were, on average, 1.56 and 1.80 times more efficient than the N = 1 SOC design to deliver. The mean aperture size produced by the RAO plans was 3.9 times larger than that of the DMS plans. Conclusions: The DAO and dose domain optimization approach enabled high quality IMRT plans using a low-complexity collimator setup. The dosimetric quality is comparable or slightly superior to conventional MLC-based IMRT plans using the same number of delivery segments. The SOC IMRT delivery efficiency can be significantly improved by increasing the leaf numbers, but the number is still significantly lower than the number of leaves in a typical MLC.

61 RADIATION PROTECTION AND DOSIMETRY↗

Early Career Award: An Enabling Computational Framework for Uncertainty Assimilation and Propagation in Complex PDE Systems: Sparse and Low-rank Techniques (Final Report)

This project aims to develop a set of new theories, algorithms, and computational tools that enable ultra-scalable representation and propagation of high-dimensional uncertainties to circumvent the issue of curse-of-dimensionality. The research components are motivated by the fact that highly scalable algorithms are needed to achieve exascale performance for uncertainty quantification and predictive simulation at extreme scales. Although these developments are generic and applicable to a wide range of high-dimensional, complex, stochastic, partial-differential-equation (PDE) systems, this research effort will focus on predictive simulation of high-energy-density lithium batteries. To create knowledge about the precise failure mechanisms of these batteries and ultimately predict their performance and reliability, a truly data-driven, multi-scale, multi-physics framework is needed that allows studying (i) dominant sources of uncertainty in the governing physical models at all scales of relevance and (ii) the interface between the interchanging physics and scales. The impact of the planned mathematical and numerical advancements in prediction of high-dimensional uncertain systems will be broad across areas of engineering and science such as climate modeling, fusion energy, combustion, and nanoscience, among others.

97 MATHEMATICS AND COMPUTING↗

Learning earthquake ground motions via conditional generative modeling

Predicting high-fidelity ground motions for future earthquakes is crucial for seismic hazard assessment and infrastructure resilience. Conventional empirical simulations suffer from sparse sensor distribution and geographically localized earthquake locations, while physics-based methods are computationally intensive and require accurate representations of Earth structures and earthquake sources. We propose an artificial intelligence (AI) spectrogram generator, Conditional Generative Modeling for Ground Motion (CGM-GM). CGM-GM leverages earthquake magnitudes and geographic coordinates of earthquakes and sensors as inputs, when postprocessed with phase information, capturing spatially continuous Fourier amplitude spectra (FAS) as well as properties such as P and S arrivals, and waveform durations, without explicit physics constraints. This is achieved through a probabilistic autoencoder that extracts latent distributions in the time-frequency domain and variational sequential models for prior and posterior distributions. We evaluate the performance of CGM-GM using small-magnitude earthquake records from the San Francisco Bay Area, a region with high seismic risks. Here, we report that CGM-GM demonstrates potential for complementing physics-based simulations and non-ergodic empirical ground motion models, as well as shows promise in seismology and beyond.

geophysics↗

Learning Distributed Geometric Koopman Operator for Sparse Networked Dynamical Systems

Koopman operator theory provides an alternative to study nonlinear networked dynamical systems by mapping the state space to an abstract higher dimensional space where the system evolution is linear. Recent works show the application of graph neural networks (GNNs) to learn state to object-centric embeddings and achieve centralized block-wise computation of Koopman operator (KO) under additional assumptions on the underlying agents properties and constraints on the KO structure. However, the computational complexity of learning the Koopman increases exponentially for networked systems where the number of possible system states grows in a combinatorial fashion with the number of nodes. The learning challenge is further amplified for sparse networks by two factors: 1) sample sparsity for learning the Koopman operator in the non-linear space, and 2) the divergence in the dynamics of individual nodes or from one subgraph to another. Our work aims to address these challenge by formulating the representation learning of networked dynamical systems into a multi-agent paradigm and learning the Koopman operator in a distributive manner. The computational as well as performance advantages of distributed Koopman is predominant for sparse networks whereas for fully connected networks, it is shown to coincide with the centralized one. The empirical study on rope system, network of oscillators and a synthetic power system show comparable and superior performance along with computational benefits with the state-of-the-art methods.

Mukherjee, Sayak↗

Polynomial meta-models with canonical low-rank approximations: Numerical insights and comparison to sparse polynomial chaos expansions

The growing need for uncertainty analysis of complex computational models has led to an expanding use of meta-models across engineering and sciences. The efficiency of meta-modeling techniques relies on their ability to provide statistically-equivalent analytical representations based on relatively few evaluations of the original model. Polynomial chaos expansions (PCE) have proven a powerful tool for developing meta-models in a wide range of applications; the key idea thereof is to expand the model response onto a basis made of multivariate polynomials obtained as tensor products of appropriate univariate polynomials. The classical PCE approach nevertheless faces the “curse of dimensionality”, namely the exponential increase of the basis size with increasing input dimension. To address this limitation, the sparse PCE technique has been proposed, in which the expansion is carried out on only a few relevant basis terms that are automatically selected by a suitable algorithm. An alternative for developing meta-models with polynomial functions in high-dimensional problems is offered by the newly emerged low-rank approximations (LRA) approach. By exploiting the tensor–product structure of the multivariate basis, LRA can provide polynomial representations in highly compressed formats. Through extensive numerical investigations, we herein first shed light on issues relating to the construction of canonical LRA with a particular greedy algorithm involving a sequential updating of the polynomial coefficients along separate dimensions. Specifically, we examine the selection of optimal rank, stopping criteria in the updating of the polynomial coefficients and error estimation. In the sequel, we confront canonical LRA to sparse PCE in structural-mechanics and heat-conduction applications based on finite-element solutions. Canonical LRA exhibit smaller errors than sparse PCE in cases when the number of available model evaluations is small with respect to the input dimension, a situation that is often encountered in real-life problems. By introducing the conditional generalization error, we further demonstrate that canonical LRA tend to outperform sparse PCE in the prediction of extreme model responses, which is critical in reliability analysis.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗