Engineering topics

Haglin, David J.

Publications and source records attributed to Haglin, David J..

9 records

DOE OSTI2021

System and method of storing and analyzing information

A system and method of storing and analyzing information is disclosed. The system includes a compiler layer to convert user queries to data parallel executable code. The system further includes a library of multithreaded algorithms, processes, and data structures. The system also includes a multithreaded runtime library for implementing compiled code at runtime. The executable code is dynamically loaded on computing elements and contains calls to the library of multithreaded algorithms, processes, and data structures and the multithreaded runtime library.

Feo, John T.↗

DOE OSTI2018

System and method of storing and analyzing information

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

Exploring DataVortex Systems for Irregular Applications

Emerging applications for data analytics and knowledge discovery typically have irregular or unpredictable communication patterns that do not scale well on parallel systems designed for traditional bulk-synchronous HPC applications. New network architectures that focus on minimizing (short) message latencies, rather than maximizing (large) transfer bandwidths, are emerging as possible alternatives to better support those applications with irregular communication patterns. We explore a system based upon one such novel network architecture, the Data Vortex interconnection network, and examine how this system performs by running benchmark code written for the Data Vortex network, as well as a reference MPI-over- Infiniband implementation, on the same cluster.

Data Vortex, High-Performance Computing, Networkin↗

DOE OSTI2017

High-Performance Data Analytics Beyond the Relational and Graph Data Models with GEMS

Graphs represent an increasingly popular data model for data-analytics, since they can naturally represent relationships and interactions between entities. Relational databases and their pure table-based data model are not well suitable to store and process sparse data. Consequently, graph databases have gained interest in the last few years and the Resource Description Framework (RDF) became the standard data model for graph data. Nevertheless, while RDF is well suited to analyze the relationships between the entities, it is not efficient in representing their attributes and properties. In this work we propose the adoption of a new hybrid data model, based on attributed graphs, that aims at overcoming the limitations of the pure relational and graph data models. We present how we have re-designed the GEMS data-analytics framework to fully take advantage of the proposed hybrid data model. To improve analysts productivity, in addition to a C++ API for applications development, we adopt GraQL as input query language. We validate our approach implementing a set of queries on net-flow data and we compare our framework performance against Neo4j. Experimental results show significant performance improvement over Neo4j, up to several orders of magnitude when increasing the size of the input data.

Data Analytics↗

NASA NTRS2004

An Artificial Intelligence Classification Tool and Its Application to Gamma-Ray Bursts

Despite being the most energetic phenomenon in the known universe, the astrophysics of gamma-ray bursts (GRBs) has still proven difficult to understand. It has only been within the past five years that the GRB distance scale has been firmly established, on the basis of a few dozen bursts with x-ray, optical, and radio afterglows. The afterglows indicate source redshifts of z=1 to z=5, total energy outputs of roughly 10(exp 52) ergs, and energy confined to the far x-ray to near gamma-ray regime of the electromagnetic spectrum. The multi-wavelength afterglow observations have thus far provided more insight on the nature of the GRB mechanism than the GRB observations; far more papers have been written about the few observed gamma-ray burst afterglows in the past few years than about the thousands of detected gamma-ray bursts. One reason the GRB central engine is still so poorly understood is that GRBs have complex, overlapping characteristics that do not appear to be produced by one homogeneous process. At least two subclasses have been found on the basis of duration, spectral hardness, and fluence (time integrated flux); Class 1 bursts are softer, longer, and brighter than Class 2 bursts (with two second durations indicating a rough division). A third GRB subclass, overlapping the other two, has been identified using statistical clustering techniques; Class 3 bursts are intermediate between Class 1 and Class 2 bursts in brightness and duration, but are softer than Class 1 bursts. We are developing a tool to aid scientists in the study of GRB properties. In the process of developing this tool, we are building a large gamma-ray burst classification database. We are also scientifically analyzing some GRB data as we develop the tool. Tool development thus proceeds in tandem with the dataset for which it is being designed. The tool invokes a modified KDD (Knowledge Discovery in Databases) process, which is described as follows.

Hakkila, Jon↗

NASA NTRS2003

How Sample Completeness Affects Gamma-Ray Burst Classification

Unsupervised pattern-recognition algorithms support the existence of three gamma-ray burst classes: class 1 (long, large-fluence bursts of intermediate spectral hardness), class 2 (short, small-fluence, hard bursts), and class 3 (soft bursts of intermediate durations and fluences). The algorithms surprisingly assign larger membership to class 3 than to either of the other two classes. A known systematic bias has been previously used to explain the existence of class 3 in terms of class 1 ; this bias allows the fluences and durations of some bursts to be underestimated, as recently shown by Hakkila et al. We show that this bias primarily affects only the longest bursts and cannot explain the bulk of the class 3 properties. We resolve the question of class 3's existence by demonstrating how samples obtained using standard trigger mechanisms fail to preserve the duration characteristics of small-peak flux bursts. Sample incompleteness is thus primarily responsible for the existence of class 3. In order to avoid this incompleteness, we show how a new, dual-timescale peak flux can be defined in terms of peak flux and fluence. The dual-timescale peak flux preserves the duration distribution of faint bursts and correlates better with spectral hardness (and presumably redshift) than either peak flux or fluence. The techniques presented here are generic and have applicability to the studies of other transient events. The results also indicate that pattern recognition algorithms are sensitive to sample completeness; this can influence the study of large astronomical databases, such as those found in a virtual observatory.

Hakkila, Jon↗

NASA NTRS2002

How Sample Completeness Affects Gamma-Ray Burst Classification

Unsupervised pattern recognition algorithms support the existence of three gamma-ray burst classes; class I (long, large fluence bursts of intermediate spectral hardness), Class II (short, small fluence, hard bursts), and class III (soft bursts of intermediate durations and fluences). The algorithms surprisingly assign larger membership to class III than to either of the other two classes. A known systematic bias has been previously used to explain the existence of class III in terms of class I; this bias allows the fluences and durations of some bursts to be underestimated. We show that this bias primarily affects only the longest bursts and cannot explain the bulk of the class III properties. We resolve the question of class III existence by demonstrating how samples obtained using standard trigger mechanisms fail to preserve the duration characteristics of small peak flux bursts: (Sample incompleteness is thus primarily responsible for the existence of class III.) In order to avoid this incompleteness, we show how a new dual timescale peak flux can be defined in terms of peak flux and fluence. The dual timescale peak flux reserves the duration distribution of faint bursts and correlates either with spectral hardness (and presumably redshift) than either peak flux or fluence. The techniques presented here are generic and have applicability to the studies of other transient events. The results also indicate that pattern recognition algorithms are sensitive to sample completeness; this can influence the study of large astronomical databases such as those found in a Virtual Observatory.

Hakkila, Jon↗

NASA NTRS2000

Gamma-Ray Burst Class Properties

Guided by the supervised pattern recognition algorithm C4.5 developed by Quinlan in 1986, we examine the three gamma-ray burst classes identified by Mukherjee et al. in 1998. C4.5 provides strong statistical support for this classification. However, with C4.5 and our knowledge of the Burst and Transient Source Experiment (BATSE) instrument, we demonstrate that class 3 (intermediate fluence, intermediate duration, soft) does not have to be a distinct source population: statistical/systematic errors in measuring burst attributes combined with the well-known hardness/intensity correlation can cause low peak flux class 1 (high fluence, long, intermediate hardness) bursts to take on class 3 characteristics naturally. Based on our hypothesis that the third class is not a distinct one, we provide rules so that future events can be placed in either class 1 or class 2 (low fluence, short, hard). We find that the two classes are relatively distinct on the basis of Band's work in 1993 on spectral parameters alpha, beta, and E (sub peak) alone. Although this does not indicate a better basis for classification, it does suggest that different physical conditions exist for class 1 and class 2 bursts. In the process of studying burst class characteristics, we identify a new bias affecting burst fluence and duration measurements. Using a simple model of how burst duration can be underestimated, we show how this fluence duration bias can affect BATSE measurements and demonstrate the type of effect it can have on the BATSE fluence versus peak flux diagram.

Hakkila, Jon↗

NASA NTRS1999

Gamma-Ray Burst Class Properties

Guided by the Supervised pattern recognition algorithm C4.5, we examine the three gamma-ray burst classes identified by Mukherjee et al. C4.5 provides strong statistical support for this classification. However, with C4.5 and our knowledge of the BATSE instrument, we demonstrate that Class 3 (intermediate fluence, intermediate duration, soft) does not have to be a distinct source population: statistical/systematic errors in measuring burst attributes combined with the well-known hardness/intensity correlation can cause low peak flux Class I (high fluence, long, intermediate hardness) bursts to take on Class 3 characteristics naturally. Based on our hypothesis that the third class is not a distinct one, we provide rules so that future events can be placed in either Class I or Class 2 (low fluence, short, hard). Using classified bursts from the BATSE 4B Catalog, we plot log(N>P) vs. log(P) curves and study spectral features of each class. We find that the two classes are relatively distinct on the basis of spectral parameters, alpha, Beta, and E(sub peak) alone. Although this does not indicate a better basis for classification, it does suggest that different physical conditions exist for Class I and Class 2 bursts. In the process of studying burst class characteristics, we identify a new bias that affects measurement of burst fluences and durations. Using a simple model of how burst duration can be underestimated, we generally characterize how this fluence duration bias affects BATSE measurements, and demonstrate the type of effect it can have on the BATSE fluence vs. peak flux diagram.

Hakkila, Jon↗