Enabling Call Path Querying in Hatchet to Identify Performance Bottlenecks in Scientific Applications
Explore the source record for details and available documents.
Computer science↗
SEARCH · Engineering Papers
Results for “path queries”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
A Selectivity based approach to Continuous Pattern Detection in Streaming Graphs
Cyber security is one of the most significant technical challenges in current times. Detecting adversarial activities, prevention of theft of intellectual properties and customer data is a high priority for corporations and government agencies around the world. Cyber defenders need to analyze massive-scale, high-resolution network flows to identify, categorize, and mitigate attacks involving net- works spanning institutional and national boundaries. Many of the cyber attacks can be described as subgraph patterns, with promi- nent examples being insider infiltrations (path queries), denial of service (parallel paths) and malicious spreads (tree queries). This motivates us to explore subgraph matching on streaming graphs in a continuous setting. The novelty of our work lies in using the subgraph distributional statistics collected from the streaming graph to determine the query processing strategy. We introduce a “Lazy Search" algorithm where the search strategy is decided on a vertex-to-vertex basis depending on the likelihood of a match in the vertex neighborhood. We also propose a metric named “Relative Selectivity" that is used to se- lect between different query processing strategies. Our experiments performed on real online news, network traffic stream and a syn- thetic social network benchmark demonstrate 10-100x speedups over selectivity agnostic approaches.
graphs, cybersecurity↗
A Selectivity based approach to Continuous Pattern Detection in Streaming Graphs
Cyber security is one of the most significant technical challenges in current times. Detecting adversarial activities, prevention of theft of intellectual properties and customer data is a high priority for corporations and government agencies around the world. Cyber defenders need to analyze massive-scale, high-resolution network flows to identify, categorize, and mitigate attacks involving networks spanning institutional and national boundaries. Many of the cyber attacks can be described as subgraph patterns, with prominent examples being insider infiltrations (path queries), denial of service (parallel paths) and malicious spreads (tree queries). This motivates us to explore subgraph matching on streaming graphs in a continuous setting. The novelty of our work lies in using the subgraph distributional statistics collected from the streaming graph to determine the query processing strategy. We introduce a ``Lazy Search" algorithm where the search strategy is decided on a vertex-to-vertex basis depending on the likelihood of a match in the vertex neighborhood. We also propose a metric named ``Relative Selectivity" that is used to select between different query processing strategies. Our experiments performed on real online news, network traffic stream and a synthetic social network benchmark demonstrate 10-100x speedups over non-incremental, selectivity agnostic approaches.
dynamic graphs↗
Path planning by querying persistent stores of trajectory segments
We introduce an algorithm for path planning (long duration) paths of dynamical systems, given a persistent object store containing suitable collections of short duration trajectory segments. We also describe experimental results from a proof-of-concept implementation of the algorithm. The basic idea is to interpret a path planning algorithm as a suitable query on a persistent object store consisting of short duration trajectory segments. The query returns a concatenation of short duration trajectory segments which is close to the desired path. The needed short duration segments are computed by using a divide and conquer algorithm to break up the original path into shorter paths; each shorter path is then matched to a nearby trajectory segment which is part of the persistent object store by using a suitable index function.
Grossman, Robert L.↗
GraQL: A Query Language for High-Performance Attributed Graph Databases
Graph databases have gained increasing interest in the last few years due to the emergence of data sources which are not easily analyzable in traditional relational models or for which a graph data model is the natural representation. In order to understand the design and implementation choices for an attributed graph database backend and query language, we have started to design our infrastructure for attributed graph databases. In this paper, we describe the design considerations of our in-memory attributed graph database system with a particular focus on the data definition and query language components.
attributed graphs↗
PCalc User's Manual
PCalc is a software tool that computes travel-time predictions, ray path geometry and model queries. This software has a rich set of features, including the ability to use custom 3D velocity models to compute predictions using a variety of geometries. The PCalc software is especially useful for research related to seismic monitoring applications.
58 GEOSCIENCES↗
Knowledge management information tool analytics with distributed database engine - 15737
The D and D KM-IT is a web-based knowledge management information tool custom built for the deactivation and decommissioning (D and D) user community. The system allows interested users to post information which goes through a custom work flow. To better improve the performance and user experience, the analysis of server logs is being conducted using MongoDB. MongoDB is a document database that provides high performance, high availability, and easy scalability. Objectives: - Feasibility study of Distributed Database Engine, MongoDB for web analytics; - Analyze server logs to evaluate D and D KM-IT module usage; - Generate useful information from the analysis. Method: - The server logs were loaded and stored in MongoDB for analysis; - Analysis and visualization was performed using NodeJS and Java Script library. Results: - It was possible to store log files in MongoDB; - Log files with different time stamps and groups from different servers can be aggregated; - Queries can be written to extract meaningful information from aggregated log files (e.g. users from different countries/organizations). Conclusions: - MongoDB is useful for server log analytics as the analyst can immediately access, analyze, and visualize MongoDB data based on the requirements; - The information can be extracted to develop custom reports; - Optimized queries can be run for fast data access. Path forward: - Custom queries need to be developed to extract information leading to improved user information.
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
pCalc User's Manual
pCalc is a software tool that computes travel-time predictions, ray path geometry and model queries. This software has a rich set of features, including the ability to use custom 3D velocity models to compute predictions using a variety of geometries. The pCalc software is especially useful for research related to seismic monitoring applications. The pCalc software is available on the web at: www.sandia.gov/salsa3d/Software.html The software is packaged with this user's manual and a set of example datasets, the use of which is described in this manual.
97 MATHEMATICS AND COMPUTING↗
Querying databases of trajectories of differential equations 2: Index functions
Suppose that a large number of parameterized trajectories (gamma) of a dynamical system evolving in R sup N are stored in a database. Let eta is contained R sup N denote a parameterized path in Euclidean space, and let parallel to center dot parallel to denote a norm on the space of paths. A data structures and indices for trajectories are defined and algorithms are given to answer queries of the following forms: Query 1. Given a path eta, determine whether eta occurs as a subtrajectory of any trajectory gamma from the database. If so, return the trajectory; otherwise, return null. Query 2. Given a path eta, return the trajectory gamma from the database which minimizes the norm parallel to eta - gamma parallel.
Grossman, Robert↗
Query Relaxation for LLM-Generated SPARQL Queries over Building Knowledge Graphs
When Knowledge Graph (KG) queries fail to match a pattern in a KG, they return no results. Identifying the statements causing these failures is tedious, especially for LLM-generated queries, which tend to be longer and more complex than queries written by hand. Query relaxation addresses this by systematically loosening query constraints until results are recovered. To evaluate the effectiveness of query relaxation against LLM generated queries, we propose a two-stage relaxation method combining triple deletion and path relaxation and test it against 1,823 failed queries for building KGs.
Paul, Lazlo↗
Propagating orientation constraints for the Hubble Space Telescope
An observing program on the Hubble Space Telescope (HST) is described in terms of exposures that are obtained by one or more of the instruments onboard the HST. Many requested exposures might specify orientation requirements and accompanying ranges. Orientation refers to the amount of roll (in degrees) about the line of sight. The range give the permissible tolerance (also in degrees). These requirements may be (1) absolute (in relation to the celestial coordinate system), (2) relative to the nominal roll angle for HST during that exposure, or (3) relative (in relation to other exposures in the observing program). The TRANSformation expert system converts proposals for astronomical observations with HST into detailed observing plans. Part of the conversion involves grouping exposures into higher level structures based on exposure characteristics. Exposures constrained to be at different orientations cannot be grouped together. Because relative orientation requirements cause implicit constraints, orientation constraints have to be propagated. TRANS must also identify any inconsistencies that may exist so they can be corrected. We have designed and implemented an orientation constraint propagator as part of TRANS. The propagator is based on an informal algebra that facilitates the setting up and propagation of the orientation constraints. The constraint propagator generates constraints between directly related exposures, and propagates derived constraints between exposures that are related indirectly. It provides facilities for path-consistency checking, identification of unsatisfiable constraints, and querying of orientation relationships. The system has been successfully operational as part of TRANS for over seven months. The solution has particular significance to space applications in which satellite/telescope pointing and attitude are constrained and relationships exist between multiple configurations.
Bose, Ashim↗
Enabling Graph Appliance for Genome Assembly
In recent years, there has been a huge growth in the amount of genomic data available as reads generated from various genome sequencers. The number of reads generated can be huge, ranging from hundreds to billions of nucleotide, each varying in size. Assembling such large amounts of data is one of the challenging computational problems for both biomedical and data scientists. Most of the genome assemblers developed have used de Bruijn graph techniques. A de Bruijn graph represents a collection of read sequences by billions of vertices and edges, which require large amounts of memory and computational power to store and process. This is the major drawback to de Bruijn graph assembly. Massively parallel, multi-threaded, shared memory systems can be leveraged to overcome some of these issues. The objective of our research is to investigate the feasibility and scalability issues of de Bruijn graph assembly on Cray s Urika-GD system; Urika-GD is a high performance graph appliance with a large shared memory and massively multithreaded custom processor designed for executing SPARQL queries over large-scale RDF data sets. However, to the best of our knowledge, there is no research on representing a de Bruijn graph as an RDF graph or finding Eulerian paths in RDF graphs using SPARQL for potential genome discovery. In this paper, we address the issues involved in representing a de Bruin graphs as RDF graphs and propose an iterative querying approach for finding Eulerian paths in large RDF graphs. We evaluate the performance of our implementation on real world ebola genome datasets and illustrate how genome assembly can be accomplished with Urika-GD using iterative SPARQL queries.
Big data↗
Swap Path Network for Robust Person Search Pre-training
This code corresponds to the WACV25 conference paper, "Swap Path Network for Robust Person Search Pre-training". In that paper, we introduce a new model for the person search task called the Swap Path Net (SPNet). The person search task is a problem in computer vision, where we locate and rank matches to an image of a query person in a set of other images where we want to find them. We also introduce a novel pre-training algorithm specific to the Swap Path Net architecture. The code implements pre-training and fine-tuning of the Swap Path Net (SPNet). This includes ingesting image datasets and updating the weights of the SPNet neural network to train it for the person search task. The repository contains code, configs, and instructions to reproduce all results from the paper.
Jaffe, LucasW [Lawrence Livermore National Laborat↗
MO-F-CAMPUS-T-05: SQL Database Queries to Determine Treatment Planning Resource Usage
Purpose: A radiation oncology clinic’s treatment capacity is traditionally thought to be limited by the number of machines in the clinic. As the number of fractions per course decrease and the number of adaptive plans increase, the question of how many treatment plans a clinic can plan becomes increasingly important. This work seeks to lay the ground work for assessing treatment planning resource usage. Methods: Care path templates were created using the Aria 11 care path interface. Care path tasks included key steps in the treatment planning process from the completion of CT simulation through the first radiation treatment. SQL Server Management Studio was used to run SQL queries to extract task completion time stamps along with care path template information and diagnosis codes from the Aria database. 6 months of planning cycles were evaluated. Elapsed time was evaluated in terms of work hours within Monday – Friday, 7am to 5pm. Results: For the 195 validated treatment planning cycles, the average time for planning and MD review was 22.8 hours. Of those cases 33 were categorized as urgent. The average planning time for urgent plans was 5 hours. A strong correlation between diagnosis code and range of elapsed planning time was as well as between elapsed time and select diagnosis codes was observed. It was also observed that tasks were more likely to be completed on the date due than the time that they were due. Follow-up confirmed that most users did not look at the due time. Conclusion: Evaluation of elapsed planning time and other tasks suggest that care paths should be adjusted to allow for different contouring and planning times for certain diagnosis codes and urgent cases. Additional clinic training around task due times vs dates or a structuring of care paths around due dates is also needed.
60 APPLIED LIFE SCIENCES↗
Querying databases of trajectories of differential equations: Data structures for trajectories
One approach to qualitative reasoning about dynamical systems is to extract qualitative information by searching or making queries on databases containing very large numbers of trajectories. The efficiency of such queries depends crucially upon finding an appropriate data structure for trajectories of dynamical systems. Suppose that a large number of parameterized trajectories gamma of a dynamical system evolving in R sup N are stored in a database. Let Eta is contained in set R sup N denote a parameterized path in Euclidean Space, and let the Euclidean Norm denote a norm on the space of paths. A data structure is defined to represent trajectories of dynamical systems, and an algorithm is sketched which answers queries.
Grossman, Robert↗
In Situ Indexing and Query Processing of AMR Data
We have developed the first, holistic, Adaptive Mesh Refinement (AMR) capable, in situ query framework to ensure the tractability of the current- and next-generation AMR simulation analysis on the path to extreme scale. We have addressed key issues pertaining to in situ analysis, memory hierarchy, generalized parallel indexing, index-compression, and real-time access pattern detection. We have published the following research papers in high profile peer reviewed computer science conferences: (a) Parallel In Situ Detection of Connected Components in Adaptive Mesh Refinement Data; (b) AMR-aware In Situ Indexing and Scalable Querying; (c) Pattern-driven dynamic data layout optimization; (d) In situ Storage Layout Optimization for AMR Spatio-temporal Read Accesses; (e) AMRZone: A Runtime AMR Data Sharing Framework for Scientific Applications; (f) Exploring Memory Hierarchy and Network Topology for Runtime AMR Data Sharing Across Scientific Applications.
96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
Streamlining Stakeholder Data Accessibility Utilizing Web-Based GIS Capabilities - 18672
Environmental data is often complex, and it can be difficult to provide proper context to fully understand and comprehensively analyze the data. One tool identified to better present complicated data is the use of web-based Geographic Information Systems (GIS). Web-based GIS provides much-needed visual context to the complex data collected at Los Alamos National Laboratory (LANL). By overlaying environmental data atop relevant geographical features, such as topography, surface or subsurface geology, hydrogeological features, buildings, roads, and technical areas, LANL's diverse group of stakeholders are better able to understand environmental data collected at the LANL site within the framework of the surrounding geography. Since 2012, Los Alamos has utilized a web-based GIS module in the Intellus New Mexico data-sharing web site. This web site provides unfettered public access to LANL and New Mexico Environment Department (NMED) - Department of Energy (DOE) Oversight Bureau environmental data. Through the past five years of real world experience, LANL identified areas where questions and confusion arose, particularly for those not directly associated with the site's environmental cleanup mission. In order to query the millions of records in Intellus in a meaningful way, users had to have substantial knowledge of the data organization and nomenclature ahead of time. While this was not a challenge for LANL and NMED environmental scientists, it proved to be a substantial barrier for members of the general public to utilize Intellus. In 2017, LANL initiated an Intellus redesign with the goal of deconstructing the type of questions people asked and identifying the types of user personas accessing the site to engineer a simpler path to desired data sets. Greatly simplified click paths ('Quick Paths') were created that walk the user through the basics with a minimum of jargon and a strong reliance on mapping results for easy visualization. The improved mapping was enabled by an upgraded online GIS functionality that allowed LANL to easily integrate their own base-maps to provide users the most up to date views of their data. The integrated Intellus GIS was enhanced to assimilate with LANL's dynamic base maps and site information in relation to data collection activities, to avoid outdated presentations. A simplified user interface (UI) was added and easier data querying was designed. A more robust visual presentation was created that allowed better viewing of spatial data. All data queries in the new simplified 'Quick Path' give the users the option to map the results and save their results to their optional accounts. This brings the power of GIS visualization to all users without the need for GIS knowledge or extensive training. By integrating with Esri{sup R} technologies, base-map layers in ArcGIS Online{sup R} can be updated by LANL or other data providers at any time and reflected immediately in the Intellus mapping application called GIS+ 1. Users can even load additional map layers to GIS+ that contain their own spatial data or information shared from over a million users on ArcGIS Online{sup R}, such as ecological information from the Fish and Wildlife Service, or geology layers from the USGS. The results from the improved quick path search in Intellus can be viewed in GIS+ and analyzed using Esri's{sup R} Smart Mapping technology, which makes it easy to classify results with its visual representation of the results in an interactive histogram, where sliders can be dragged to look at various intervals for each color range for that query. With enhanced GIS+, Los Alamos National Laboratory is able to observe improve Stakeholder engagement and positive Stakeholder interactions within its public environmental database, Intellus New Mexico. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Spatial and Temporal Deconfliction of Trajectories in the Presence of Uncertainties
Demonstration of conflict-free movement for multi-agent teams in challenging scenarios is crucial in developing trust and trustworthiness in an autonomous transport system. Tolerance verification queries are explored as a mechanism to enforce spatial and temporal deconfliction for a cooperating team of Unmanned Aerial Systems (UAS) with prescribed heterogeneous path-following performance. Obstacles in the environment are modelled as set of polyhedra, whereas each vehicle’s trajectory is represented as a sequence of polynomial curves with C2 continuity, expressed in a Bernstein basis. Each vehicle is modelled as a point mass and a safety distance, informed by the geometry of the UAS and the worst-case path-following error. This defines a tube around the trajectories where each UAS is most likely to fly through. In addition, obstacles in the environment have an associated safety buffer around them to account for the uncertainty in their location and geometric description. The tolerance verification queries explored in this paper combine the safety distance information from each UAS and environmental hazard to compute trajectories that are contained within the safe configuration space. Tolerance verification is also compared with other proximity queries to determine the suitability of each method along the different steps of the trajectory generation algorithm. This paper analyzes the fitness and performance of three proximity queries – collision, tolerance verification, and distance computations – between polyhedral and polynomial curves to ensure deconfliction between obstacles and vehicles, but also between polynomial curves to guarantee safe separation among cooperating UAS.
trajectory generation↗