Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scalable Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Remote Sensing of Tundra Ecosystems Using High Spectral Resolution Reflectance: Opportunities and Challenges

Abstract Observing the environment in the vast regions of Earth through remote sensing platforms provides the tools to measure ecological dynamics. The Arctic tundra biome, one of the largest inaccessible terrestrial biomes on Earth, requires remote sensing across multiple spatial and temporal scales, from towers to satellites, particularly those equipped for imaging spectroscopy (IS). We describe a rationale for using IS derived from advances in our understanding of Arctic tundra vegetation communities and their interaction with the environment. To best leverage ongoing and forthcoming IS resources, including National Aeronautics and Space Administration’s Surface Biology and Geology mission, we identify a series of opportunities and challenges based on intrinsic spectral dimensionality analysis and a review of current data and literature that illustrates the unique attributes of the Arctic tundra biome. These opportunities and challenges include thematic vegetation mapping, complicated by low‐stature plants and very fine‐scale surface composition heterogeneity; development of scalable algorithms for retrieval of canopy and leaf traits; nuanced variation in vegetation growth and composition that complicates detection of long‐term trends; and rapid phenological changes across brief growing seasons that may go undetected due to low revisit frequency or be obscured by snow cover and clouds. We recommend improvements to future field campaigns and satellite missions, advocating for research that combines multi‐scale spectroscopy, from lab studies to satellites that enable frequent and continuous long‐term monitoring, to inform statistical and biophysical approaches to model vegetation dynamics.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Non-Gaussian estimates of tensions in cosmological parameters

We discuss how to efficiently and reliably estimate the level of agreement and disagreement on parameter determinations from different experiments, fully taking into account non-Gaussianities in the parameter posteriors. We develop two families of scalable algorithms that allow us to perform this type of calculations in increasing number of dimensions and for different levels of tensions. One family of algorithms rely on kernel density estimates of posterior distributions while the other relies on machine learning modeling of the posterior distribution with normalizing flows. We showcase their effectiveness and accuracy with a set of benchmark examples and find both methods agree with each other and the true tension within 0.5σ in difficult cases and generally to 0.2σ or better. This allows us to study the level of internal agreement between different measurements of the clustering of cosmological structures from the Dark Energy Survey and their agreement with measurements of the Cosmic Microwave Background from the Planck satellite.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

A Scalable Distributed Louvain Algorithm for Large-Scale Graph Community Detection

We present a new distributed community detection algorithm for large graphs based on the Louvain method. We exploit a distributed delegate partitioning to ensure the workload and communication balancing among processors. In addition, we design a new heuristic strategy to carefully coordinate the community constitution in a distributed environment, and ensure the convergence of the distributed clustering algorithm. Our intensive experimental study has demonstrated the scalability and the correctness of our algorithm with various large-scale real-world and synthetic graph datasets using up to 32,768 processors.

Computer Science↗
DOE OSTI2018

A Study of Clustering Techniques and Hierarchical Matrix Formats for Kernel Ridge Regression

We present memory-efficient and scalable algorithms for kernel methods used in machine learning. Using hierarchical matrix approximations for the kernel matrix the memory requirements, the number of floating point operations, and the execution time are drastically reduced compared to standard dense linear algebra routines. We consider both the general H matrix hierarchical format as well as Hierarchically Semi-Separable (HSS) matrices. Furthermore, we investigate the impact of several preprocessing and clustering techniques on the hierarchical matrix compression. Effective clustering of the input leads to a ten-fold increase in efficiency of the compression. The algorithms are implemented using the STRUMPACK solver library. These results confirm that - with correct tuning of the hyperparameters - classification using kernel ridge regression with the compressed matrix does not lose prediction accuracy compared to the exact - not compressed - kernel matrix and that our approach can be extended to O(1M) datasets, for which computation with the full kernel matrix becomes prohibitively expensive. We present numerical experiments in a distributed memory environment up to 1,024 processors of the NERSC's Cori supercomputer using well-known datasets to the machine learning community that range from dimension 8 up to 784.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Computationally Efficient Decompositions of Oblique Projection Matrices

Oblique projection matrices arise in problems in weighted least squares, signal processing, and optimization. While these matrices can be potentially very large, their low-rank structure can be exploited for efficient computation. Here, we propose fast and scalable algorithms for computing their eigendecomposition and singular value decomposition (SVD). Numerical experiments that compare our proposed approaches to existing methods, including randomized SVD, are presented. In addition, we test their accuracy on linear systems from equality constrained optimization problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

DELTACON: A Principled Massive-Graph Similarity Function with Attribution

How much did a network change since yesterday? How different is the wiring between Bob's brain (a left-handed male) and Alice's brain (a right-handed female)? Graph similarity with known node correspondence, i.e. the detection of changes in the connectivity of graphs, arises in numerous settings. In this work, we formally state the axioms and desired properties of the graph similarity functions, and evaluate when state-of-the-art methods fail to detect crucial connectivity changes in graphs. We propose DeltaCon, a principled, intuitive, and scalable algorithm that assesses the similarity between two graphs on the same nodes (e.g. employees of a company, customers of a mobile carrier). In our experiments on various synthetic and real graphs we showcase the advantages of our method over existing similarity measures. We also employ DeltaCon to real applications: (a) we classify people to groups of high and low creativity based on their brain connectivity graphs, and (b) do temporal anomaly detection in the who-emails-whom Enron graph.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Oak Ridge Graph Analytics for Medical Innovation (ORiGAMI)

In this era of data-driven decisions and discovery where Big Data is producing Bigger Data, data scientists at the Oak Ridge National Laboratory are leveraging unique leadership infrastructure (e.g., Urika XA and Urika GD appliances) to develop scalable algorithms for semantic, logical and statistical reasoning with Big Data (i.e., data stored in databases as well as unstructured data in documents). ORiGAMI is a next-generation knowledge-discovery framework that is: (a) knowledge nurturing (i.e., evolves seamlessly with newer knowledge and data), (b) smart and curious (i.e. using information-foraging and reasoning algorithms to digest content) and (c) synergistic (i.e., interfaces computers with what they do best to help subject-matter-experts do their best. ORiGAMI has been demonstrated using the National Library of Medicine's SEMANTIC MEDLINE (archive of medical knowledge since 1994).

Roberts, LarryW.↗
DOE OSTI2021

pnnl/pakman

PaKman: A Scalable Algorithm for Generating Genomic Contigs on Distributed Memory Machines. PaKman presents a fully distributed method that tackles assembly of large genomes through the combinationof a novel data-structure (PaK-Graph) and algorithmic strategies to simplify communication and I/O footprint during the assembly process.

Ghosh, Priyanka↗
DOE OSTI2026

BoBa

BoBa is a C++ software library for working with large matrices, tensors, and tensor decompositions. The library provides tools for dense matrix and tensor operations, tensor decompositions, and tensor decomposition methods that support modern CPU and GPU architectures. It includes portable abstractions for linear algebra, tensor algebra, and multidimensional computation. BoBa is intended for scientific computing applications that involve large multidimensional data sets or high dimensional mathematical models. Its capabilities support tasks such as data compression, linear algebra, efficient numerical computation, and the development of scalable algorithms for heterogeneous hardware. Tutorials, tests, and example applications are included to help users learn and apply the library.

Yao, Jin [Lawrence Livermore National Laboratory (↗
DOE OSTI2014

Computation Annual Report (2013)

Computation provides the intellectual electricity that helps make Lawrence Livermore National Laboratory (LLNL) a thriving and innovative research and development (R&D) institution. Our hardware, software, and discipline experts are integrated into almost every field of research at LLNL. Our in-house programs, software applications, and support initiatives help achieve mission-critical national-security objectives for the U.S. Departments of Energy, Homeland Security, and Defense, as well as other federal and state agencies. We deliver comprehensive support to a wide set of scientific disciplines that include physical sciences (materials modeling, fluid dynamics), biology (genomic sequencing), mathematics (scalable algorithms), and computer sciences (parallel processing, real-time control systems). This year, we have expanded our outreach with high performance computing (HPC) collaborations and made great strides in our capabilities related to mathematical computation, control systems, and computation infrastructure, many of them in concert with partners in industry and academia. The Computation workforce remains highly motivated and passionate about melding computational expertise with cutting-edge science and technology to enable breakthroughs. 2013 has once again proven that Computation provides the foundation and support that are essential for productivity at LLNL. Every scientific discipline and program leverages the tools and expertise provided by our staff. We remain at the forefront of HPC moving towards exascale and have developed the infrastructure and proficiency to make this an achievable goal. Our researchers repeatedly deliver outstanding research, attracting collaborations with the biggest names in academia and industry, while joining forces with other national laboratories to further push the bounds of what is scientifically possible. Computation’s mission is echoed by the President of the United States in his assertion that “our future depends on reaffirming America’s role as the world’s engine of scientific discovery and technological innovation.” Driven by this lofty goal, we employ our state-of-the-art computational capabilities to ensure LLNL’s position as a global science and technology leader.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Rutgers University Subcontract B611610 Final Report

Given an incomplete (i.e., partially-observed) network, which nodes should we actively probe in order to achieve the highest accuracy for a given network feature? For example, consider a cyber-network administrator who observes only a portion of the network at time t and wants to accurately identify the most important (e.g., highest PageRank) nodes in the complete network. She has a limited budget for probing the network. Of all the nodes she has observed, which should she probe in order to most accurately identify the important nodes? We propose a novel and scalable algorithm, MaxOutProbe, and evaluate it w.r.t. four network features (largest connected component, PageRank, core-periphery, and community detection), five network sampling strategies, and seven network datasets from different domains. Across a range of conditions, MaxOutProbe demonstrates consistently high performance relative to several baseline strategies

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Final technical report: SciDAC partnership "Predictive Computing for Condensed Matter"

Fundamental progress in energy technology depends critically on our ability to design, fabricate, characterize, and optimize advanced materials that can convert, transport, and store energies as well as display useful functions at minimum energy consumptions. To achieve this, we must first understand mechanisms underlying the functions and then be able to design materials with predicted properties. Unlike conventional structural materials, optoelectronic or magnetic materials for energy technology operate by quantum mechanical principles, which are hard to extract from experiments alone. Some representative unsolved problems of condensed-phase electronic structures in general include (1) the detailed mechanism of high-temperature superconductors such as cuprates and iron-arsenide materials; (2) the structures of solids in the presence of polymorphism; (3) excited states in solids and systematically accurate calculations of quasiparticle energy bands; (4) the structure of strongly correlated wave functions of solids. In this project, we aim at establishing general and predictive mathematical theories for solids and computational methods that implement them. We will then apply these methods to outstanding solid-state problems recognized by experimentalists including the above, with a view to solving them computationally. Hence, the project has the following three research objectives, all of which have been met: Predictive Electronic Structure Theories for Solids, Scalable Algorithms and Software, and Applications to Outstanding Condensed Matter Problems.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2018

Modeling Mesoscale Processes of Scalable Synthesis

Brown University is part of the Collaboratory on Mathematics for Mesoscopic Modeling of Materials (CM4), led by PNNL. The Collaboratory focuses on developing rigorous mathematical foundations for understanding and controlling fundamental mechanisms in mesoscale processes to enable scalable synthesis of complex materials, through the design of efficient modeling methods and corresponding scalable algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Sparse Recovery for Scientific Data

We have built build and analyzed fundamental mathematical tools that enable sparse recovery of scientific data for high performance computing, experimental facilities and sensor measurement. The proposed work will impact many areas of research across the DOE. We have designed, developed and analyzed fast scalable algorithms for the sparse recovery of scientific data sets that may be extremely large, generated very quickly and distributed across multiple storage devices. The algorithms we developed will be used to process the copious amounts of scientific data generated by the DOE supercomputer simulations and by DOE experimental and observational facilities. They leverage the fact that while the amount of data produced can be extremely large, the actual quantities of interest (QoI's) are often sparse, either in space time, or in some feature space. Mathematically, notions of compressive sensing, including l1 regularized and total variation regularized optimization have played an important part in this effort.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Scalable Mixed-Integer Decomposition Method for Security-Constrained Optimal Power Flow with Complementarity Constraints

This project aimed to develop a scalable algorithm for security-constrained optimal power flow (SCOPF) under contingency scenarios. In particular, the SCOPF problem targeted in the GO Competition is challenging because of the nonconvexity, its nonsmoothness, and the problem size, which increases with the number of contingency events. Complementarity constraints imposed in post-contingency variables are particularly challenging because they lead to a violation of constraint qualifications at any feasible point.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Theoretical framework for new magnetic materials for quantum computing and information storage. Final report for the Award No. DE-SC0018910

The focus of this grant was on molecular magnetic materials for information storage and quantum computing. We have been developing robust, first-principle methods for computing relevant electronic and magnetic properties of molecular building blocks (SMMs) of novel magnetic materials and quantum computers. These tools enable theoretical modeling of SMMs’ behavior, facilitating the interpretation of experimental studies and aiding the design of novel magnetic materials. Our strategy is based on the spin-flip (SF) approach, which extends the hierarchy of black-box single-reference methods to strongly correlated systems. Specifically, we developed general scalable algorithms and computer codes for calculating molecular properties, with an emphasis on spin-related properties, such as zero-field splittings, hyperfine couplings, and g-tensors. While our primary focus was on SF wave functions and SF-TDDFT, the underlying theory and computer codes were formulated using reduced density matrices, such that these tools are applicable to a broader class of methods. To extend the scope of applicability of wave-function-based SF methods to larger systems, we developed reduced-scaling approaches for the equation-of-motion coupled-cluster (EOM-CC) methods and continue developing libtensor (our open-source general tensor contraction library for many-body methods). We carried out extensive benchmarks and also carried out several applications.

36 MATERIALS SCIENCE↗
DOE OSTI2022

High Fidelity CFD Simulations Supporting the KP-FHR

Kairos Power, LLC, is developing its version of the Fluoride-cooled High-temperature Reactor, the KP-FHR. The design uses a pebble bed core with fluoride salt as a coolant. The pebbles used in the KP-FHR have a diameter of 4 cm, with a shell fuel region where TRISO particles are embedded. A Pebble bed core design is adopted by several Gen IV reactors, They boast many benefits, such as fuel integrity, highly efficient heat transfer, and passive safety. However, it is challenging to accurately predict temperature and flow inside a pebble bed. Traditional approaches use the porous media model, which regards the pebble bed as a continuous medium, but with different temperature fields representing different levels, such as the fluid temperature, pebble surface temperature, and pebble center temperature. Empirical heat transfer correlations are adopted to calculate the heat transfer coefficient between different phases. However, empirical correlations are usually validated with experimental data, which usually lacks detail inside the pebble bed. The available experimental data is also generally at a high Reynolds number, which falls outside of the conditions of KP-FHR. Explicit computational fluid dynamics (CFD) simulations of randomly packed pebble beds have only become feasible recently. This is thanks to the rapid development of computational power and scalable algorithms. In this work, we used the Spectral Element Method (SEM) CFD code NekRS to simulate the randomly packed pebble bed in a cylindrical container. NekRS, which is the GPU variant of Nek5000, but refactored to utilize the computational power of GPUs using the OCCA library to run on hybrid architecture high performance computing systems. It was initially developed with the libParamunal library, but truncated and tuned for large-scale turbulence simulation. As a result, the SEM reaches higher precision with the same degrees of freedom by using a high-order Lagrange polynomial basis distributed on Gauss-Lobatto-Legendre quadrature inside each element, compared to lower-order methods, such the Finite Volume Method and Finite Element Method. The report is divided into five parts. We start with a general discussion of the pebble bed reactor, along with a specific investigation into the KP-FHR. The second part presents the numerical methodology. In the third part, we study a modular pebble bed with 1741 pebbles in a container of 7 pebble-diameter radius. Beyond LES simulations done by NekRS, we also leveraged the thermal radiation model in OpenFOAM to study heat transfer under no-forced-flow scenarios. Then, in the fourth part we simulated a pebble bed similar to the size of the Hermes Test Reactor. The total number of pebbles is in these simulations is 34,374. The container radius is 14 pebble-diameters. Finally, the report concludes in part five, with a discussion of future work.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗