Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sparse data set”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Adaptive pruning-based optimization of parameterized quantum circuits

Abstract Variational hybrid quantum–classical algorithms are powerful tools to maximize the use of noisy intermediate-scale quantum devices. While past studies have developed powerful and expressive ansatze, their near-term applications have been limited by the difficulty of optimizing in the vast parameter space. In this work, we propose a heuristic optimization strategy for such ansatze used in variational quantum algorithms, which we call ‘parameter-efficient circuit training (PECT)’. Instead of optimizing all of the ansatz parameters at once, PECT launches a sequence of variational algorithms, in which each iteration of the algorithm activates and optimizes a subset of the total parameter set. To update the parameter subset between iterations, we adapt the Dynamic Sparse Reparameterization scheme which was originally proposed for training deep convolutional neural networks. We demonstrate PECT for the Variational Quantum Eigensolver, in which we benchmark unitary coupled-cluster ansatze including UCCSD and k -UpCCGSD, as well as the Low-Depth Circuit Ansatz (LDCA), to estimate ground state energies of molecular systems. We additionally use a layerwise variant of PECT to optimize a hardware-efficient circuit for the Sycamore processor to estimate the ground state energy densities of the one-dimensional Fermi-Hubbard model. From our numerical data, we find that PECT can enable optimizations of certain ansatze that were previously difficult to converge and more generally can improve the performance of variational algorithms by reducing the optimization runtime and/or the depth of circuits that encode the solution candidate(s).

Physics↗
DOE OSTI2019

Early Career Award: An Enabling Computational Framework for Uncertainty Assimilation and Propagation in Complex PDE Systems: Sparse and Low-rank Techniques (Final Report)

This project aims to develop a set of new theories, algorithms, and computational tools that enable ultra-scalable representation and propagation of high-dimensional uncertainties to circumvent the issue of curse-of-dimensionality. The research components are motivated by the fact that highly scalable algorithms are needed to achieve exascale performance for uncertainty quantification and predictive simulation at extreme scales. Although these developments are generic and applicable to a wide range of high-dimensional, complex, stochastic, partial-differential-equation (PDE) systems, this research effort will focus on predictive simulation of high-energy-density lithium batteries. To create knowledge about the precise failure mechanisms of these batteries and ultimately predict their performance and reliability, a truly data-driven, multi-scale, multi-physics framework is needed that allows studying (i) dominant sources of uncertainty in the governing physical models at all scales of relevance and (ii) the interface between the interchanging physics and scales. The impact of the planned mathematical and numerical advancements in prediction of high-dimensional uncertain systems will be broad across areas of engineering and science such as climate modeling, fusion energy, combustion, and nanoscience, among others.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Variance Preserving Spectral Subsampling

Generating statistically faithful short-duration gamma-ray spectra from a single long measurement is essential in nuclear safeguards, supporting tasks such as algorithm development and machine-learning applications, especially when list-mode data are unavailable. Existing subsampling methods often distort the statistical characteristics of genuine short-duration measurements, leading to biased or unreliable analytical outcomes and thereby undermining downstream tasks. In this work, we compare five subsampling approaches using a benchmark set of 156 genuine replicate spectra collected with a high-purity germanium detector. We evaluate each method with respect to run-to-run variance, channel-to-channel variance, and preservation of total counts (losslessness). Across a wide range of subsampling ratios, only binomial subsampling without replacement consistently reproduces the statistical properties of genuine short-duration spectra, maintaining proper dispersion even in sparse spectral regions and perfectly preserving total counts. These results provide a mathematically principled and practically validated framework for generating synthetically shortened spectra when true short-duration measurements are unavailable.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2023

Forward sensitivity analysis and mode dependent control for closure modeling of Galerkin systems

Model reduction by projection-based approaches is often associated with losing some of the important features that contribute towards the dynamics of the retained scales. As a result, a mismatch occurs between the predicted trajectories of the original system and the truncated one. We put forth a framework to apply a continuous time control signal in the latent space of the reduced order model (ROM) to account for the effect of truncation. We set the control input using parameterized models by following energy transfer principles. Our methodology relies on observing the system behavior in the physical space and using the projection operator to restrict the feedback signal into the latent space. Then, we leverage the forward sensitivity method (FSM) to derive relationships between the feedback and the desired mode-dependent control. We test the performance of the proposed approach using two test cases, corresponding to viscous Burgers and vortex merger problems at high Reynolds number. Results show that the ROM trajectory with the applied FSM control closely matches its target values in both the data-dense and data-sparse regimes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Shifting Between Compute and Memory Bounds: A Compression-Enabled Roofline Model

In the evolving landscape of high-performance computing, especially to fight the end of Moore’s Law and Dennard’s Scaling, the ability to shift between compute-bound and memory-bound states is critical for enhancing adaptability and flexibility to diverse system and domain-specific architectures. Such capability is vital for optimizing performance across distinguished hardware configurations, such as accelerators, memory hierarchies, and cache systems. Despite that ad hoc optimization techniques, such as compressed/approximate computation, have been enabled for compute-/data-intensive computing for improved performance in distinct hardware settings, there lacks an understanding of 1) the rational behind performance improvement; 2) capability of different optimizations; 3) what optimization to respond to specific computational and memory demands. This work proposes a compression-enabled roofline model to facilitate this adaptability with data compression techniques to balance and transform between computational and memory demands. This model enables applications to adjust in response to the specific strengths and limitations of the underlying hardware and system to optimize resource utilization. The effectiveness of this approach is demonstrated with matrix multiplication kernels on different input sizes, with turning on/off various compression techniques, including 1) low-precision floating point; 2) sparse matrix formulation; and 3) compressed arrays with ZFP. By reducing memory transfer volumes and cache misses and increasing data locality and computational intensity through compression, the specific roofline model can transform between compute and memory bounds to align more efficiently with system capabilities. This advancement not only improves overall performance but also maximizes adaptability in diverse computing environments.

Naraparaju, Ramasoumya [University of Washington]↗
DOE OSTI2017

MPI-FAUN: An MPI-Based Framework for Alternating-Updating Nonnegative Matrix Factorization

Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors W and H, for the given input matrix A, such that A≈WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient parallel algorithms to solve the problem for big data sets. The main contribution of this work is a new, high-performance parallel computational framework for a broad class of NMF algorithms that iteratively solves alternating non-negative least squares (NLS) subproblems for W and H. It maintains the data and factor matrices in memory (distributed across processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild assumptions). The framework is flexible and able to leverage a variety of NMF and NLS algorithms, including Multiplicative Update, Hierarchical Alternating Least Squares, and Block Principal Pivoting. Our implementation allows us to benchmark and compare different algorithms on massive dense and sparse data matrices of size that spans from few hundreds of millions to billions. We demonstrate the scalability of our algorithm and compare it with baseline implementations, showing significant performance improvements. The code and the datasets used for conducting the experiments are available online.

2D↗
DOE OSTI2016

An advanced joint inversion system for CO 2 storage modeling with large date sets for characterization and real-time monitoring-enhancing storage performance and reducing failure risks under uncertainties

As large-scale, commercial storage projects become operational, the problem of utilizing information from diverse sources becomes more critically important. In this project, we developed, tested, and applied an advanced joint data inversion system for CO 2 storage modeling with large data sets for use in site characterization and real-time monitoring. Emphasis was on the development of advanced and efficient computational algorithms for joint inversion of hydro-geophysical data, coupled with state-of-the-art forward process simulations. The developed system consists of (1) inversion tools using characterization data, such as 3D seismic survey (amplitude images), borehole log and core data, as well as hydraulic, tracer and thermal tests before CO 2 injection, (2) joint inversion tools for updating the geologic model with the distribution of rock properties, thus reducing uncertainty, using hydro-geophysical monitoring data, and (3) highly efficient algorithms for directly solving the dense or sparse linear algebra systems derived from the joint inversion. The system combines methods from stochastic analysis, fast linear algebra, and high performance computing. The developed joint inversion tools have been tested through synthetic CO 2 storage examples.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2016

Characterizing spatiotemporal information loss in sparse-sampling-based dynamic MRI for monitoring respiration-induced tumor motion in radiotherapy

Purpose: Sparse-sampling and reconstruction techniques represent an attractive strategy to achieve faster image acquisition speeds, while maintaining adequate spatial resolution and signal-to-noise ratio in rapid magnetic resonance imaging (MRI). The authors investigate the use of one such sequence, broad-use linear acquisition speed-up technique (k-t BLAST) in monitoring tumor motion for thoracic and abdominal radiotherapy and examine the potential trade-off between increased sparsification (to increase imaging speed) and the potential loss of “true” information due to greater reliance on a priori information. Methods: Lung tumor motion trajectories in the superior–inferior direction, previously recorded from ten lung cancer patients, were replayed using a motion phantom module driven by an MRI-compatible motion platform. Eppendorf test tubes filled with water which serve as fiducial markers were placed in the phantom. The modeled rigid and deformable motions were collected in a coronal image slice using balanced fast field echo in conjunction with k-t BLAST. Root mean square (RMS) error was used as a metric of spatial accuracy as measured trajectories were compared to input data. The loss of spatial information was characterized for progressively increasing acceleration factor from 1 to 16; the resultant sampling frequency was increased approximately from 2.5 to 19 Hz when the principal direction of the motion was set along frequency encoding direction. In addition to the phantom study, respiration-induced tumor motions were captured from two patients (kidney tumor and lung tumor) at 13 Hz over 49 s to demonstrate the impact of high speed motion monitoring over multiple breathing cycles. For each subject, the authors compared the tumor centroid trajectory as well as the deformable motion during free breathing. Results: In the rigid and deformable phantom studies, the RMS error of target tracking at the acquisition speed of 19 Hz was approximately 0.3–0.4 mm, which was smaller than the reconstructed pixel resolution of 0.67 mm. In the patient study, the dynamic 2D MRI enabled the monitoring of cycle-to-cycle respiratory variability present in the tumor position. It was seen that the range of centroid motion as well as the area covered due to target motion during each individual respiratory cycle was underestimated compared to the entire motion range observed over multiple breathing cycles. Conclusions: The authors’ initial results demonstrate that sparse-sampling- and reconstruction-based dynamic MRI can be used to achieve adequate image acquisition speeds without significant information loss for the task of radiotherapy guidance. Such monitoring can yield spatial and temporal information superior to conventional offline and online motion capture methods used in thoracic and abdominal radiotherapy.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Physics informed deep neural network embedded in a chemical transport model for the Amazon rainforest

Secondary organic aerosols (SOA) are fine particles in the atmosphere, which interact with clouds, radiation and affect the Earth’s energy budget. SOA formation involves chemistry in gas phase, aqueous aerosols, and clouds. Simulating these chemical processes involve solving a stiff set of differential equations, which are computationally expensive steps for three-dimensional chemical transport models. Deep neural networks (DNNs) are universal function approximators that could be used to represent the complex nonlinear changes in aerosol physical and chemical processes; however, key challenges such as generalizability to extended time periods, preservation of mass balance, simulating sparse model outputs, and maintaining physical constraints have limited their use in atmospheric chemistry. Here, we develop an approach of using a physics-informed DNN that overcomes previous such challenges and demonstrates its applicability for the chemical formation processes of isoprene epoxydiol SOA (IEPOX-SOA) over the Amazon rainforest. The DNN is trained with data generated by simulating IEPOX-SOA over the entire atmospheric column, using the Weather Research and Forecasting Model coupled with Chemistry (WRF-Chem). The trained DNN is then embedded within WRF-Chem to replace the computationally expensive default solver of IEPOX-SOA formation. The trained DNN predictions generalizes well with the default model simulation of the IEPOX-SOA mass concentrations and its size distribution (20 size bins) over several days of simulations in both dry and wet seasons. The embedded DNN reduces the computational expense of WRF-Chem by a factor of 2. Our approach shows promise in terms of application to other computationally expensive chemistry solvers in climate models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Analysis of the Temporal Response of Coupled Asymmetrical Zero-Power Subcritical Bare Metal Reactor Systems

The behavior of symmetrical coupled-core systems has been extensively studied, yet there is a dearth of research on asymmetrical systems due to the increased complexity of the analysis of such systems. In this research, the multipoint kinetics method is applied to asymmetrical zeropower, subcritical, bare metal reactor systems. Existing research on asymmetrical reactor systems assumes symmetry in the neutronic coupling; however, it will be shown that this cannot always be assumed. Deep subcriticality adds another layer of complexity and requires modification of the multipoint kinetics equations to account for the effect of the external neutron source. A modified set of multipoint kinetics equations is derived with this in mind. Subsequently, the Rossi-alpha equations are derived for a two-region asymmetrical reactor system. The predictive capabilities of the radiation transport code MCNP6 for neutron noise experiments are shown in a comparison to the results of a series of Rossi-alpha measurements performed by J. Mihalczo utilizing a coupled set of symmetrical bare highly-enriched uranium (HEU) cylinders. The ptrac option within MCNP6 can generate time-tagged counts in a cell (list-mode data). The list-mode data can then be processed similarly to measured data to obtain values for system parameters such as the dual prompt neutron decay constants observable in a coupled system. The results from the ptrac simulations agree well with the historical measured values. A series of case studies are conducted to study the effects of geometrical asymmetry in the coupling between two bare metal HEU cylinders. While the coupling behavior of symmetrical systems has been reported on extensively, that of asymmetrical systems remains sparse. In particular, it appears that there has been no previous research in obtaining the coupling time constants for asymmetrically-coupled systems. The difficulty in observing such systems is due in part to the inability to determine the individual coupling coefficients from measurement: unlike the symmetrical cases, only the product of the values can be obtained. A method is proposed utilizing MCNP6 tally ratios to separate the coupling coefficients for such systems. This work provides insight into the behavior of asymmetrically-coupled systems as the separation distance between the two cores is changed and also as the asymmetry is increased. As the asymmetry increases, both the slower and the faster observable prompt neutron decay constants increase in magnitude. The coupling time constants are determined from the measured decay constants. As the separation distance increases, both coupling coefficients decrease as expected. Based on these findings, an effective computational method utilizing MCNP6 and the Rossialpha technique can be applied to the prediction of asymmetrical coupled system measurements.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

To Fail or not to Fail: An Exploration of Machine Learning Techniques for Predictive Maintenance

Predictive maintenance refers to the ability to predict when machinery or systems need to be maintained. Making an accurate prediction is quite challenging given the costs for both over-estimating (unnecessary maintenance and reduction in availability of assets) and under-estimating (untimely breakdowns and possible loss of equipment or lives). To address these challenges researchers were able to develop new approaches for analyzing oil samples taken extracting samples from oil-wetted machinery that may provide information critical to developing predictive capabilities. We consider the problem from both supervised (though data limited) and unsupervised approaches and provide a first look into a data driven approach for identification of condition indicators. Through this work we identify a collection of candidate features that can form the basis of condition indicators for both a high level discrimination of failure vs. normal operation as well as a set for potential failure mode identification. Finally, we present an anomaly detection framework for detecting failures which can be a viable solution for an onboard analysis tool in deployed systems.

predictive maintenance, anomaly detection, Laserne↗
DOE OSTI2024

Using automated machine learning for the upscaling of gross primary productivity

Estimating gross primary productivity (GPP) over space and time is fundamental for understanding the response of the terrestrial biosphere to climate change. Eddy covariance flux towers provide in situ estimates of GPP at the ecosystem scale, but their sparse geographical distribution limits larger-scale inference. Machine learning (ML) techniques have been used to address this problem by extrapolating local GPP measurements over space using satellite remote sensing data. However, the accuracy of the regression model can be affected by uncertainties introduced by model selection, parameterization, and choice of explanatory features, among others. Recent advances in automated ML (AutoML) provide a novel automated way to select and synthesize different ML models. In this work, we explore the potential of AutoML by training three major AutoML frameworks on eddy covariance measurements of GPP at 243 globally distributed sites. We compared their ability to predict GPP and its spatial and temporal variability based on different sets of remote sensing explanatory variables. Explanatory variables from only Moderate Resolution Imaging Spectroradiometer (MODIS) surface reflectance data and photosynthetically active radiation explained over 70 % of the monthly variability in GPP, while satellite-derived proxies for canopy structure, photosynthetic activity, environmental stressors, and meteorological variables from reanalysis (ERA5-Land) further improved the frameworks' predictive ability. We found that the AutoML framework Auto-sklearn consistently outperformed other AutoML frameworks as well as a classical random forest regressor in predicting GPP but with small performance differences, reaching an r 2 of up to 0.75. We deployed the best-performing framework to generate global wall-to-wall maps highlighting GPP patterns in good agreement with satellite-derived reference data. This research benchmarks the application of AutoML in GPP estimation and assesses its potential and limitations in quantifying global photosynthetic activity.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Quantification of spatial and seasonal trends in the atmosphere and construction of statistical models for infrasonic propagation

SUMMARY Infrasonic waves are influenced by variations in the density, pressure and temperature as well as the ambient winds. Modelling infrasonic propagation can be challenging due to the dynamic nature of the atmosphere as well as the sparseness of measurements which result in variability and notable uncertainty. A framework is presented to quantify spatial and seasonal trends in atmospheric structure via analysis of the effective sound speed profile and identification of temporal trends in the middle atmospheric waveguide produced by the circumpolar vortex winds. Seasonal definitions identifying typical atmospheric structures during the summer, winter and spring/fall transition periods are identified using atmospheric data from 2010 through 2020. Seasonal trend analysis is conducted for a number of locations across the contiguous United States to quantify spatial variations in atmospheric structure that impact infrasonic propagation. For each season and location, empirical orthogonal function analysis is used to reduce the historical archive of atmospheric data into a smaller representative set that can be analysed using numerical tools more efficiently. Infrasonic ray tracing and finite-frequency modal propagation analyses are applied to construct propagation path geometry and transmission loss statistics which are useful in localization and yield estimation for infrasonic sources, respectively. An example application is detailed in which transmission loss statistics are combined with an explosive source model and noise statistics to quantify the capability of a network to detect nearby sources.

58 GEOSCIENCES↗
DOE OSTI2019

Data Integration Aggregated Model and Ontology for Nuclear Deployment (DIAMOND): Preliminary Model and Ontology

The data of a nuclear utility or plant are stored in various isolated forms in different systems. These data have different structures and tools and are, therefore, used independently. The integration of data sources is often performed manually and on an as-needed basis to achieve a targeted cost saving. A holistic approach to the integration of the data would enable creating a data warehouse for a single nuclear power plant (NPP) or multiple plants. A data warehouse results in direct and indirect cost saving because it automates the manual search for data, enables sharing and comparison of data from various tools of a single or multiple plants, enable digital transformation of data, increase frequencies of needed data that are sparse in nature (such as failures signatures), reduces the need for training on various tools for plant staff, enables a holistic staff perception of plant activities, reveals cost-saving opportunities, and improves the visual perception of all parts a plant necessary for operations and maintenance. One of the main challenges to deploy a data warehouse is associated with the integration of data sources into a consistent data model using a standard ontology that is developed by and for the nuclear power industry. This model is non-existent and is a necessity to enable the nuclear data warehouse role. A data model contains objects that make up a given domain along with their attributes and relationships. It provides domain standardization and maintainability, setting forth the objects, attributes, and relationships within that domain and acting as the guide by which the various applications within that domain will structure their data and interchange. An ontology is defined as “a set of concepts and categories in a subject area or domain that shows their properties and the relations between them”. Ontologies consist of a taxonomy or directed rooted tree. A survey of the models and ontologies available to support a data integration aggregated model and ontology for nuclear deployment (DIAMOND) was performed and different standards that address specific aspects of the needed model and ontology were leveraged and integrated in this effort. In specific, the Basic Formal Ontology (BFO) and Lifecycle Modeling Language (LML) were leveraged for the base ontology. The ISO 4224 and 15926 were leveraged in the development of the overlapping nuclear and fossil data aspects. The electric power research institute (EPRI) power distribution common information model (CIM) was also reviewed and the experience gained was leveraged in the power generation scope of DIAMOND. The initial aim targeted by this effort was to develop the ontology definitions and model structure. This report documents the resulting model and ontology to develop DIAMOND. The report describes the various elements of the model development based on a growing tree development approach, i.e. DIAMOND’s tree structure is established with the core data aspects and is populated with a preliminary level of details. Fifteen sources of data in a nuclear power plant were considered and incorporated into DIAMOND at this stage of development, representing the majority of operations data sources in nuclear power plants. Additional data sources, scope, and a higher level of fidelity will be incorporated into DIAMOND next in collaboration with the nuclear industry and relevant stakeholders.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Machine learning methods for probabilistic locked-mode predictors in tokamak plasmas

A rotating tokamak plasma can interact resonantly with the external helical magnetic perturbations, also known as error fields. This can lead to locking and then to disruptions. We leverage machine learning (ML) methods to predict the locking events. We use a coupled third-order nonlinear ordinary differential equation model to represent the interaction of the magnetic perturbation and the plasma rotation with the error field. This model is sufficient to describe qualitatively the locking and unlocking bifurcations. Here, we explore using ML algorithms with the simulation data and experimental data, focusing on the methods that can be used with sparse datasets. These methods lead to the possibility of the avoidance of locking in real-time operations. We describe the operational space in terms of two control parameters: the magnitude of the error field and the rotation frequency associated with the momentum source that maintains the plasma rotation. The outcomes are quan- tified by order parameters that completely characterize the state, whether locked or unlocked. We use unsupervised ML methods to classify locked/unlocked states and note the usefulness of a certain normalization of the order parameters. Three supervised ML classifiers are used in suite to estimate the probability of locking in the region of control parameter space with hysteresis, i.e., the set of control parameters for which both locked and unlocked states can exist. The results show that a neural network gives the best estimate of the locking probability. An analogy of the present locking model with the van der Waals equation of state is also provided.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2016

PERIOD ESTIMATION FOR SPARSELY SAMPLED QUASI-PERIODIC LIGHT CURVES APPLIED TO MIRAS

We develop a nonlinear semi-parametric Gaussian process model to estimate periods of Miras with sparsely sampled light curves. The model uses a sinusoidal basis for the periodic variation and a Gaussian process for the stochastic changes. We use maximum likelihood to estimate the period and the parameters of the Gaussian process, while integrating out the effects of other nuisance parameters in the model with respect to a suitable prior distribution obtained from earlier studies. Since the likelihood is highly multimodal for period, we implement a hybrid method that applies the quasi-Newton algorithm for Gaussian process parameters and search the period/frequency parameter space over a dense grid. A large-scale, high-fidelity simulation is conducted to mimic the sampling quality of Mira light curves obtained by the M33 Synoptic Stellar Survey. The simulated data set is publicly available and can serve as a testbed for future evaluation of different period estimation methods. The semi-parametric model outperforms an existing algorithm on this simulated test data set as measured by period recovery rate and quality of the resulting period–luminosity relations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Parameter Sensitivity Analysis of the SparTen High Performance Sparse Tensor Decomposition Software (Extended Analysis)

Tensor decomposition models play an increasingly important role in modern data science applications. One problem of particular interest is fitting a low-rank Canonical Polyadic (CP) tensor decomposition model when the tensor has sparse structure and the tensor elements are nonnegative count data. SparTen is a high-performance C++ library which computes a low-rank decomposition using different solvers: a first-order quasi-Newton or a second-order damped Newton method, along with the appropriate choice of runtime parameters. Since default parameters in SparTen are tuned to experimental results in prior published work on a single real-world dataset conducted using MATLAB implementations of these methods, it remains unclear if the parameter defaults in SparTen are appropriate for general tensor data. Furthermore, it is unknown how sensitive algorithm convergence is to changes in the input parameter values. This report addresses these unresolved issues with large-scale experimentation on three benchmark tensor data sets. Experiments were conducted on several different CPU architectures and replicated with many initial states to establish generalized profiles of algorithm convergence behavior.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

High-Performance Data Analytics Beyond the Relational and Graph Data Models with GEMS

Graphs represent an increasingly popular data model for data-analytics, since they can naturally represent relationships and interactions between entities. Relational databases and their pure table-based data model are not well suitable to store and process sparse data. Consequently, graph databases have gained interest in the last few years and the Resource Description Framework (RDF) became the standard data model for graph data. Nevertheless, while RDF is well suited to analyze the relationships between the entities, it is not efficient in representing their attributes and properties. In this work we propose the adoption of a new hybrid data model, based on attributed graphs, that aims at overcoming the limitations of the pure relational and graph data models. We present how we have re-designed the GEMS data-analytics framework to fully take advantage of the proposed hybrid data model. To improve analysts productivity, in addition to a C++ API for applications development, we adopt GraQL as input query language. We validate our approach implementing a set of queries on net-flow data and we compare our framework performance against Neo4j. Experimental results show significant performance improvement over Neo4j, up to several orders of magnitude when increasing the size of the input data.

Data Analytics↗