Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Queries”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

Prevalence of LuxR- and LuxI-type quorum sensing circuits in members of the Populus deltoides microbiome

We are interested in the root microbiome of the fast-growing Eastern cottonwood tree, Populus 25 deltoides. There is a large bank of bacterial isolates from P. deltoides and there are 44 draft 26 genomes of bacterial endophyte and rhizosphere isolates. As a first step in efforts to understand 27 the roles of bacterial communication and plant-bacterial signaling in P. deltoides we focused on 28 the prevalence of acyl-homoserine lactone (AHL) quorum sensing signal production and 29 reception in members of the P. deltoides microbiome. We screened 129 bacterial isolates for 30 AHL production using a broad-spectrum bioassay that responds to many but not all AHLs, and 31 we queried the available genome sequences of microbiome isolates for homologs of AHL 32 synthase and receptor genes. AHL signal production was detected in 40% of 129 strains tested. 33 Positive isolates included -, - and -Proteobacteria. Members of the luxI family of AHL 34 synthases were identified in 18 of 39 Proteobacteria genomes including genomes of some 35 isolates that tested negative in the bioassay. Members of the luxR family of transcription factors, 36 that include AHL-responsive factors, were more abundant than luxI homologs. There were 72 in 37 the 39 Proteobacteria genomes. Some of the luxR homologs appear to be members of a 38 subfamily of LuxRs that respond to as yet unknown plant signals rather than bacterial AHLs. 39 Apparently, there is a substantial capacity for AHL cell-to-cell communication in Proteobacteria 40 of the P. deltoides microbiota and there are also Proteobacteria with LuxR homologs of the type 41 hypothesized to respond to plant signals or cues.

Schaefer, Amy L↗

Quorum Quenching of Nitrobacter winogradskyi Suggests that Quorum Sensing Regulates Fluxes of Nitrogen Oxide(s) during Nitrification

ABSTRACT Quorum sensing (QS) is a widespread process in bacteria used to coordinate gene expression with cell density, diffusion dynamics, and spatial distribution through the production of diffusible chemical signals. To date, most studies on QS have focused on model bacteria that are amenable to genetic manipulation and capable of high growth rates, but many environmentally important bacteria have been overlooked. For example, representatives of proteobacteria that participate in nitrification, the aerobic oxidation of ammonia to nitrate via nitrite, produce QS signals called acyl-homoserine lactones (AHLs). Nitrification emits nitrogen oxide gases (NO, NO 2 , and N 2 O), which are potentially hazardous compounds that contribute to global warming. Despite considerable interest in nitrification, the purpose of QS in the physiology/ecology of nitrifying bacteria is poorly understood. Through a quorum quenching approach, we investigated the role of QS in a well-studied AHL-producing nitrite oxidizer, Nitrobacter winogradskyi . We added a recombinant AiiA lactonase to N. winogradskyi cultures to degrade AHLs to prevent their accumulation and to induce a QS-negative phenotype and then used mRNA sequencing (mRNA-Seq) to identify putative QS-controlled genes. Our transcriptome analysis showed that expression of nirK and nirK cluster genes ( ncgABC ) increased up to 19.9-fold under QS-proficient conditions (minus active lactonase). These data led to us to query if QS influenced nitrogen oxide gas fluxes in N. winogradskyi . Production and consumption of NO x increased and production of N 2 O decreased under QS-proficient conditions. Quorum quenching transcriptome approaches have broad potential to identify QS-controlled genes and phenotypes in organisms that are not genetically tractable. IMPORTANCE Bacterial cell-cell signaling, or quorum sensing (QS), is a method of bacterial communication and gene regulation that is well studied in bacteria. However, little is known about the purpose of QS in many environmentally important bacteria. Here, we demonstrate quorum quenching coupled with mRNA-Seq to identify QS-controlled genes and phenotypes in Nitrobacter winogradskyi , a nitrite-oxidizing bacterium. Nitrite oxidizers play an important role in the nitrogen cycle though their participation in nitrification, the aerobic oxidation of ammonia to nitrate via nitrite. Our quorum quenching approach revealed that QS influences production and consumption of environmentally important nitrogen oxide gases (NO, NO 2 , and N 2 O) in N. winogradskyi . This study demonstrated a novel technique for studying QS in difficult-to-work-with microorganisms and showed that nitrite oxidizers might also contribute to nitrification-dependent production of nitrogen oxide gases that contribute to global warming.

Mellbye, Brett L.↗

Curated BLAST for Genomes

Given a microbe’s genome sequence, we often want to predict what capabilities the organism has, such as which nutrients it requires or which energy sources it can use. Or, we know the organism has a capability and we want to find the genes involved. Scientists often use automated gene annotations to find relevant genes, but automated annotations are often vague or incorrect. Curated BLAST finds candidate genes for a capability without relying on automated annotations. First, Curated BLAST finds proteins (usually from other organisms) whose functions have been studied experimentally and whose curated descriptions match a query. Then, it searches the genome of interest for similar proteins and returns a list of candidates. Curated BLAST is fast and often finds relevant genes that are missed by automated annotation.

59 BASIC BIOLOGICAL SCIENCES↗

Toward a Better Understanding of Species Interactions through Network Biology

Within the last decade, there has been an explosion of multi-omics data generated for several microbial systems. At the same time, new methods of analysis have emerged that are based on inferring networks that link features both within and between species based on correlation in abundance. These developments prompt two important questions. What can be done with network approaches to better understand microbial species interactions? What challenges remain in applying network approaches to query the more complex systems of natural settings? Here, I briefly describe what has been done and what questions still need to be answered. Over the next 5 to 10 years, we will be in a strong position to infer networks that contain multiple kinds of omic data and describe systems with multiple species. These applications will open the door for a better understanding and use of microbiomes across a variety of fields.

Microbiology↗

Native Plasmid-Encoded Mercury Resistance Genes Are Functional and Demonstrate Natural Transformation in Environmental Bacterial Isolates

Plasmid-mediated horizontal gene transfer (HGT) is a major driver of genetic diversity in bacteria. We experimentally validated the function of a putative mercury resistance operon present on an abundant 8-kbp native plasmid found in groundwater samples without detectable levels of mercury. Phylogenetic analyses of the plasmid-encoded mercury reductases from the studied groundwater site show them to be distinct from those reported in proximal metal-contaminated sites. We synthesized the entire native plasmid and demonstrated that the plasmid was sufficient to confer functional mercury resistance in Escherichia coli . Given the possibility that natural transformation is a prevalent HGT mechanism in the low-cell-density environments of groundwaters, we also assayed bacterial strains from this environment for competence. We used the native plasmid-encoded metal resistance to design a screen and identified 17 strains positive for natural transformation. We selected 2 of the positive strains along with a model bacterium to fully confirm HGT via natural transformation. From an ecological perspective, the role of the native plasmid population in providing advantageous traits combined with the microbiome’s capacity to take up environmental DNA enables rapid adaptation to environmental stresses. Horizontal transfer of mobile genetic elements via natural transformation has been poorly understood in environmental microbes. Here, we confirm the functionality of a native plasmid-encoded mercury resistance operon in a model microbe and then query for the dissemination of this resistance trait via natural transformation into environmental bacterial isolates. We identified 17 strains including Gram-positive and Gram-negative bacteria to be naturally competent. These strains were able to successfully take up the plasmid DNA and obtain a clear growth advantage in the presence of mercury. Our study provides important insights into gene dissemination via natural transformation enabling rapid adaptation to dynamic stresses in groundwater environments.

59 BASIC BIOLOGICAL SCIENCES↗

Geometry Helps to Compare Persistence Diagrams

Exploiting geometric structure to improve the asymptotic complexity of discrete assignment problems is a well-studied subject. In contrast, the practical advantages of using geometry for such problems have not been explored. We implement geometric variants of the Hopcroft--Karp algorithm for bottleneck matching (based on previous work by Efrat el al.), and of the auction algorithm by Bertsekas for Wasserstein distance computation. Both implementations use k-d trees to replace a linear scan with a geometric proximity query. Our interest in this problem stems from the desire to compute distances between persistence diagrams, a problem that comes up frequently in topological data analysis. We show that our geometric matching algorithms lead to a substantial performance gain, both in running time and in memory consumption, over their purely combinatorial counterparts. Moreover, our implementation significantly outperforms the only other implementation available for comparing persistence diagrams.

97 MATHEMATICS AND COMPUTING↗

LR: Compact connectivity representation for triangle meshes

We propose LR (Laced Ring) - a simple data structure for representing the connectivity of manifold triangle meshes. LR provides the option to store on average either 1.08 references per triangle or 26.2 bits per triangle. Its construction, from an input mesh that supports constant-time adjacency queries, has linear space and time complexity, and involves ordering most vertices along a nearly-Hamiltonian cycle. LR is best suited for applications that process meshes with fixed connectivity, as any changes to the connectivity require the data structure to be rebuilt. We provide an implementation of the set of standard random-access, constant-time operators for traversing a mesh, and show that LR often saves both space and traversal time over competing representations.

97 MATHEMATICS AND COMPUTING↗

Distributed merge trees

Improved simulations and sensors are producing datasets whose increasing complexity exhausts our ability to visualize and comprehend them directly. To cope with this problem, we can detect and extract significant features in the data and use them as the basis for subsequent analysis. Topological methods are valuable in this context because they provide robust and general feature definitions. As the growth of serial computational power has stalled, data analysis is becoming increasingly dependent on massively parallel machines. To satisfy the computational demand created by complex datasets, algorithms need to effectively utilize these computer architectures. The main strength of topological methods, their emphasis on global information, turns into an obstacle during parallelization. We present two approaches to alleviate this problem. We develop a distributed representation of the merge tree that avoids computing the global tree on a single processor and lets us parallelize subsequent queries. To account for the increasing number of cores per processor, we develop a new data structure that lets us take advantage of multiple shared-memory cores to parallelize the work on a single node. Finally, we present experiments that illustrate the strengths of our approach as well as help identify future challenges. Copyright © 2013 ACM 978-1-4503-1922-5/13/ 02...$15.00.

Morozov, D↗

Enabling the High Level Synthesis of Data Analytics Accelerators

Conventional High Level Synthesis (HLS) tools mainly tar- get compute intensive kernels typical of digital signal pro- cessing applications. We are developing techniques and ar- chitectural templates to enable HLS of data analytics appli- cations. These applications are memory intensive, present fine-grained, unpredictable data accesses, and irregular, dy- namic task parallelism. We discuss an architectural tem- plate based around a distributed controller to efficiently ex- ploit thread level parallelism. We present a memory in- terface that supports parallel memory subsystems and en- ables implementing atomic memory operations. We intro- duce a dynamic task scheduling approach to efficiently ex- ecute heavily unbalanced workload. The templates are val- idated by synthesizing queries from the Lehigh University Benchmark (LUBM), a well know SPARQL benchmark.

Data Analytics↗

DART: distributed adaptive radix tree for efficient affix-based keyword search on HPC systems

© 2018 Association for Computing Machinery. Affix-based search is a fundamental functionality for storage systems. It allows users to find desired datasets, where attributes of a dataset match an affix. While building inverted index to facilitate efficient affix based keyword search is a common practice for standalone databases and for desktop file systems, building local indexes or adopting indexing techniques used in a standalone data store is insufficient for highperformance computing (HPC) systems due to the massive amount of data and distributed nature of the storage devices within a system. In this paper, we propose Distributed Adaptive Radix Tree (DART), to address the challenge of distributed affix-based keyword search on HPC systems. This trie-based approach is scalable in achieving efficient affix-based search and alleviating imbalanced keyword distribution and excessive requests on keywords at scale. Our evaluation at different scales shows that, comparing with the "full string hashing" use case of the most popular distributed indexing technique - Distributed Hash Table (DHT), DART achieves up to 55× better throughput with prefix search and with suffix search, while achieving comparable throughput with exact and infix searches. Also, comparing to the "initial hashing" use case of DHT, DART maintains a balanced keyword distribution on distributed nodes and alleviates excessive query workload against popular keywords.

Zhang, Wei↗

One SQL to Rule Them All - an Efficient and Syntactically Idiomatic Approach to Management of Streams and Tables

Real-time data analysis and management are increasingly critical for today's businesses. SQL is the de facto lingua franca for these endeavors, yet support for robust streaming analysis and management with SQL remains limited. Many approaches restrict semantics to a reduced subset of features and/or require a suite of non-standard constructs. Additionally, use of event timestamps to provide native support for analyzing events according to when they actually occurred is not pervasive, and often comes with important limitations. We present a three-part proposal for integrating robust streaming into SQL, namely: (1) time-varying relations as a foundation for classical tables as well as streaming data, (2) event time semantics, (3) a limited set of optional keyword extensions to control the materialization of time-varying query results. We show how with these minimal additions it is possible to utilize the complete suite of standard SQL semantics to perform robust stream processing. We motivate and illustrate these concepts using examples and describe lessons learned from implementations in Apache Calcite, Apache Flink, and Apache Beam. We conclude with syntax and semantics of a concrete proposal for extensions of the SQL standard and note further areas of exploration.

Begoli, Edmon↗

PETSc DMNetwork: A Library for Scalable Network PDE-Based Multiphysics Simulations

We present DMNetwork, a high-level package included in the PETSc library for the simulation of multiphysics phenomena over large-scale networked systems. The library aims at applications that have networked structures such as those in electrical, gas, and water distribution systems. DMNetwork provides data and topology management, parallelization for multiphysics systems over a network, and hierarchical and composable solvers to exploit the problem structure. DMNetwork eases the simulation development cycle by providing the necessary infrastructure through simple abstractions to define and query the network components. This article presents the design of DMNetwork, illustrates its user interface, and demonstrates its ability to solve multiphysics systems, such as an electric circuit, a network of power grid and water subnetworks, and transient hydraulic systems over large networks with more than 2 billion variables on extreme-scale computers using up to 30,000 processors.

extreme-scale computing↗

ArborX: A Performance Portable Geometric Search Library

Searching for geometric objects that are close in space is a fundamental component of many applications. The performance of search algorithms comes to the forefront as the size of a problem increases both in terms of total object count as well as in the total number of search queries performed. Scientific applications requiring modern leadership-class supercomputers also pose an additional requirement of performance portability, i.e., being able to efficiently utilize a variety of hardware architectures. In this article, we introduce a new open-source C++ search library, ArborX, which we have designed for modern supercomputing architectures. Herein, we examine scalable search algorithms with a focus on performance, including a highly efficient parallel bounding volume hierarchy implementation, and propose a flexible interface making it easy to integrate with existing applications. We demonstrate the performance portability of ArborX on multi-core CPUs and GPUs and compare it to the state-of-the-art libraries such as Boost.Geometry.Index and nanoflann.

97 MATHEMATICS AND COMPUTING↗

Scalable Pattern Matching in Metadata Graphs via Constraint Checking

Pattern matching is a fundamental tool for answering complex graph queries. Unfortunately, existing solutions have limited capabilities: They do not scale to process large graphs and/or support only a restricted set of search templates or usage scenarios. Moreover, the algorithms at the core of the existing techniques are not suitable for today’s graph processing infrastructures relying on horizontal scalability and shared-nothing clusters, as most of these algorithms are inherently sequential and difficult to parallelize. In this article we present an algorithmic pipeline that bases pattern matching on constraint checking. The key intuition is that each vertex and edge participating in a match has to meet a set of constraints implicitly specified by the search template. These constraints can be verified independently and typically are less expensive to compute than searching the full template. The pipeline we propose generates these constraints and iterates over them to eliminate all the vertices and edges that do not participate in any match, thus reducing the background graph to a subgraph that is the union of all template matches—the complete set of all vertices and edges that participate in at least one match. Additional analysis can be performed on this annotated, reduced graph, such as full match enumeration, match counting, or computing vertex/edge centrality. Furthermore, a vertex-centric formulation for constraint checking algorithms exists, and this makes it possible to harness existing high-performance, vertex-centric graph processing frameworks. This technique (i) enables highly scalable pattern matching in metadata (labeled) graphs; (ii) supports arbitrary patterns with 100% precision; (iii) enables tradeoffs between precision and time-to-solution, while always selects all vertices and edges that participate in matches, thus offering 100% recall; and (iv) supports a set of popular data analytics scenarios. We implement our approach on top of HavoqGT, an open-source asynchronous graph processing framework, and demonstrate its advantages through strong and weak scaling experiments on massive scale real-world (up to 257 billion edges) and synthetic (up to 4.4 trillion edges) labeled graphs, respectively, and at scales (1,024 nodes / 36,864 cores), orders of magnitude larger than used in the past for similar problems. This article serves two purposes: First, it synthesises the knowledge accumulated during a long-term project. Second, it presents new system features, usage scenarios, optimizations, and comparisons with related work that strengthen the confidence that pattern matching based on iterative pruning via constraint checking is an effective and scalable approach in practice. The new contributions include the following: (i) We demonstrate the ability of the constraint checking approach to efficiently support two additional search scenarios that often emerge in practice, interactive incremental search and exploratory search. (ii) We empirically compare our solution with two additional state-of-the-art systems, Arabsque and TriAD. (iii) We show the ability of our solution to accommodate a more diverse range of datasets with varying properties, e.g., scale, skewness, label distribution, and match frequency. (iv) We introduce or extend a number of system features (e.g., work aggregation, load balancing, and the ability to cap the generated traffic) and design optimizations and demonstrate their advantages with respect to improving performance and scalability. (v) We present bottleneck analysis and insights into artifacts that influence performance. (vi) We present a theoretical complexity argument that motivates the performance gains we observe.

97 MATHEMATICS AND COMPUTING↗

HBMax: Optimizing Memory Efficiency for Parallel Influence Maximization on Multicore Architectures

The goal of influence maximization is to select k most-influential vertices or seeds in a network, where influence is defined by a given diffusion process. The problem has a number of important applications such as viral marketing, information spread, and epidemic control. Although computing optimal seed set is NP-Hard, due to the submodular nature of the problem efficient approximation algorithms exist. However, even state-of-the-art parallel implementations are limited by a sampling step that incurs large memory footprints. This in turn limits the problem size reach and approximation quality. In this work, we study the memory footprint of the sampling process collecting reverse reachability information in the IMM algorithm over large real-world social networks. We present an adaptive and memory-efficient optimization approach for a state-of-the-art multi-threaded parallel influence maximization algorithm. Our approach,HuffMax, uses a portion of the reverse reachable (RR) sets collected by the algorithm to learn the characteristics of the graph. Then, it compresses the intermediate reverse reachability information with Huffman coding, and queries directly on the compressed data to preserve the memory savings obtained through compression. We also propose an efficient sampling strategy based on the distribution of RR sets, which can further reduce the computation time for typical social networks with long-tail distributions. Considering a NUMA architecture, we scale up our solution on 128-core CPUs and reduce the memory footprint by up to 45.7% with negligible time overhead (or even faster) and without perceivable loss of accuracy.

Chen, Xinyu↗

Properly Learning Decision Trees in almost Polynomial Time

We give an n O (log log n ) -time membership query algorithm for properly and agnostically learning decision trees under the uniform distribution over { ± 1} n . Even in the realizable setting, the previous fastest runtime was n O (log n ) , a consequence of a classic algorithm of Ehrenfeucht and Haussler. Our algorithm shares similarities with practical heuristics for learning decision trees, which we augment with additional ideas to circumvent known lower bounds against these heuristics. To analyze our algorithm, we prove a new structural result for decision trees that strengthens a theorem of O’Donnell, Saks, Schramm, and Servedio. While the OSSS theorem says that every decision tree has an influential variable, we show how every decision tree can be “pruned” so that every variable in the resulting tree is influential.

Computer Science↗

Software-Hardware Co-design of Heterogeneous SmartNIC System for Recommendation Models Inference and Training

Deep Learning Recommendation Models (DLRMs) are critical applications in various domains and have evolved as one of the single largest machine learning applications. Trillions of DLRM parameters exceed the on-chip memory capacity of GPUs. Large-scale multi-node systems are required for distributed DLRM inference and training, which suffer from the all-to-all communication bottleneck, mainly limiting the scalability of ever-growing DLRMs. In recent years, SmartNICs have evolved with coupled computation and communication capabilities providing opportunities for a powerful heterogeneous device in the system. However, there isn't such a distributed system that fully leverages the abundant smartNIC resources that resolve the scalability issue of DLRMs. In this work, we proposed a software-hardware co-design of a heterogeneous smartNIC system that resolves the communication bottleneck of distributed DLRMs, mitigates the memory bandwidth pressure, and improves computation efficiency. We provide a set of smartNIC designs of cache systems (including local cache and remote cache) and smartNIC computation kernels which reduce data movement, relieve memory lookup intensity, and improve the GPU's computation efficiency. In addition, we propose a graph algorithm that improves the data locality of queries within batches which optimizes the overall system performance with higher data reuse. Our evaluation shows that our system achieves 2.1x latency speedup for inference and 1.6x throughput speedup for training.

Guo, Anqi↗

Fast tree-based algorithms for DBSCAN for low-dimensional data on GPUs

DBSCAN is a well-known density-based clustering algorithm to discover arbitrary shape clusters. While conceptually simple in serial, the algorithm is challenging to efficiently parallelize on manycore GPU architectures. Common pitfalls, such as asynchronous range query calls, result in high thread execution divergence in many implementations. In this paper, we propose a new framework for GPU-accelerated DBSCAN, and describe two tree-based algorithms within that framework. Both algorithms fuse the search for neighbors with updating cluster information, but differ in their treatment of dense regions of the data. We show that the time taken to compute clusters is at most twice that of determination of the neighbors. We compare the proposed algorithms with existing CPU and GPU implementations, and demonstrate their competitiveness and performance using a fast traversal structure (bounding volume hierarchy) for low dimensional data. We also show that the memory usage can be reduced by processing object neighbors dynamically without storing them.

Prokopenko, Andrey↗