Engineering Papers⌕ Search

Engineering topics

Begoli, Edmon

Publications and source records attributed to Begoli, Edmon.

48 records · Page 3

An Emerging Role for Polystores in Precision Medicine

Medical data is organically heterogeneous, and it usually varies significantly in both size and composition. Yet, this data is also a key for the recent and promising field of precision medicine, which focuses on identifying and tailoring appropriate medical treatments for the needs of the individual patients, based on their specific conditions, their medical history, lifestyle, genetic, and other individual factors. As we, and a database community at large, recognize that a “one size does not fit all” solution is required to work with such data, we present in this paper our observations based on our experiences, and the applications in the field of precision medicine. Finally, we make the case for the use of polystore architecture; how it applies for precision medicine; we discuss the reference architecture; describe some of its critical components (array database); and discuss the specific types of analysis that directly benefit from this database architecture, and the ways it serves the data.

60 APPLIED LIFE SCIENCES↗

Storage and Read-Optimized Data Placement Structures for High-Performance Analysis [Book Chapter]

Here, we present state-of-art structures and methods for efficient data preparation and representation for analysis. Our intent is to introduce the data science and analytics communities to open source data placements, structures, and methods. These practices can make the foundational processes of data preparation and access dramatically more efficient than typical raw file or database representations and use more conservative storage. To illustrate this, we introduce two highly efficient data placement structures. We then present a tutorial, supported by step-by-step examples, of how to create, use, and access data, structured by Parquet or ORC (Optimized Row Columnar), using Apache Spark. Finally, we illustrate the benefits of using these structures with computational and storage volume benchmarks.

97 MATHEMATICS AND COMPUTING↗

Towards a Heterogeneous, Polystore-like Data Architecture for the US Department of Veteran Affairs (VA) Enterprise Analytics

The Polystore architecture revisits the federated approach to access and querying of the standalone, independent databases in the uniform and optimized fashion, but this time in the context of heterogeneous data and specialized analyses. In the light of this architectural philosophy, and in the light of the major data architecture development efforts at the US Department of Veterans Administration (VA), we discuss the need for the heterogeneous data store consisting of the large relational data warehouse, an image and text datastore, and a peta-scale genomic repository. The VA's heterogeneous datastore would, to a larger or smaller degree, follow the architectural blueprint proposed by the polystore architecture. To this end, we discuss the current state of the data architecture at VA, architectural alternatives for development of the heterogeneous datastore, the anticipated challenges, and the drawbacks and benefits of adopting the polystore architecture.

Federated Databases↗

Real-Time Discovery Services over Large, Heterogeneous and Complex Healthcare Datasets Using Schema-Less, Column-Oriented Methods

We present a service platform for schema-leess exploration of data and discovery of patient-related statistics from healthcare data sets. The architecture of this platform is motivated by the need for fast, schema-less, and flexible approaches to SQL-based exploration and discovery of information embedded in the common, heterogeneously structured healthcare data sets and supporting components (electronic health records, practice management systems, etc.) The motivating use cases described in the paper are clinical trials candidate discovery, and a treatment effectiveness analysis. Following the use cases, we discuss the key features and software architecture of the platform, the underlying core components (Apache Parquet, Drill, the web services server), and the runtime profiles and performance characteristics of the platform. We conclude by showing dramatic speedup with some approaches, and the performance tradeoffs and limitations of others.

Apache Drill Apache Parquet Column Oriented Stores↗

Hierarchical Cluster Analysis of Service Usage and Geographic Variation in Medicare Spending

Per capita Medicare spending varies considerably throughout the Unites States, even after taking into account factors such as cost of living and pre-existing conditions of the Medicare populations. Additionally, per capita spending may vary up to four times as much under Medicare as it does under private insurance. Several studies have explored the Medicare-costs differences between regions with respect to how Medicare services are used. The recent public release of a large, relatively fine-grained set of Medicare data over four annual periods (2007–2010) by the Centers for Medicare and Medicaid Services (CMS), allowed us to perform a broad-scale exploratory analysis of Medicare service-usage patterns. The CMS data set contains data on the Health Care Referral Region level, which subsets the United States into 306 distinct regions. To identify potential trends in service usage over these regions, we used hierarchical agglomerative cluster analysis (HACA) to group regions into clusters based on similarity of service-usage patterns. Much previous work on the subject of cost variation starts with analysis of cost. By beginning with service-usage data, HACA effectively uncoupled patterns of service-usage from overall patterns of cost. We found that while some Medicare services were used in a fairly consistent manner over all clusters, other services varied considerably. Variation in how frequently services are used may provide insight into practices that are particularly expensive or particularly cost effective. It also may be possible to develop recommendations for more cost-efficient high-quality medical care by taking such information into account. In addition, the variance among the frequencies with which some services are used may point to inconsistent standards of medical practice that are not necessarily directly reflected in total cost, or to more deeply underlying geographic or demographic factors that are not necessarily reflected directly in total cost.

Vacha, Linnet↗

Performance implications from sizing a VM on multi-core systems: A Data analytic application s view

In this paper, we present a quantitative performance analysis of data analytics applications running on multi-core virtual machines. Such environments form the core of cloud computing. In addition, data analytics applications, such as Cassandra and Hadoop, are becoming increasingly popular on cloud computing platforms. This convergence necessitates a better understanding of the performance and cost implications of such hybrid systems. For example, the very rst step in hosting applications in virtualized environments, requires the user to con gure the number of virtual processors and the size of memory. To understand performance implications of this step, we benchmarked three Yahoo Cloud Serving Benchmark (YCSB) workloads in a virtualized multi-core environment. Our measurements indicate that the performance of Cassandra for YCSB workloads does not heavily depend on the processing capacity of a system, while the size of the data set is critical to performance relative to allocated memory. We also identi ed a strong relationship between the running time of workloads and various hardware events (last level cache loads, misses, and CPU migrations). From this analysis, we provide several suggestions to improve the performance of data analytics applications running on cloud computing environments.

Lim, Seung-Hwan↗

Design Principles for Effective Knowledge Discovery from Big Data

Big data phenomenon refers to the practice of collection and processing of very large data sets and associated systems and algorithms used to analyze these massive datasets. Architectures for big data usually range across multiple machines and clusters, and they commonly consist of multiple special purpose sub-systems. Coupled with the knowledge discovery process, big data movement offers many unique opportunities for organizations to benefit (with respect to new insights, business optimizations, etc.). However, due to the difficulty of analyzing such large datasets, big data presents unique systems engineering and architectural challenges. In this paper, we present three sys- tem design principles that can inform organizations on effective analytic and data collection processes, system organization, and data dissemination practices. The principles presented derive from our own research and development experiences with big data problems from various federal agencies, and we illustrate each principle with our own experiences and recommendations.

Architecture↗

Big Data Platforms as a Service: Challenges and Approach

Infrastructure-as-a-Service has revolutionized the manner in which users commission computing infrastructure. Coupled with Big Data platforms (Hadoop, Cassandra), IaaS has democratized the ability to store and process massive datasets. For users that need to customize or create new Big Data stacks, however, readily available solutions do not yet exist. Users must first acquire the necessary cloud computing infrastructure, and manually install the prerequisite software. For complex distributed services this can be a daunting challenge. To address this issue, we argue that distributed services should be viewed as a single application consisting of virtual machines. Users should no longer be concerned about individual machines or their internal organization. To illustrate this concept, we introduce Cloud-Get, a distributed package manager that enables the simple installation of distributed services in a cloud computing environment. Cloud-Get enables users to instantiate and modify distributed services, including Big Data services, using simple commands. Cloud-Get also simplifies creating new distributed services via standardized package definitions.

99 GENERAL AND MISCELLANEOUS↗

A Short Survey on the State of the Art in Architectures and Platforms for Large Scale Data Analysis and Knowledge Discovery from Data

Intended as a survey for practicing architects and researchers seeking an overview of the state-of-the-art architectures for data analysis, this paper provides an overview of the emerg- ing data management and analytic platforms including par- allel databases, Hadoop-based systems, High Performance Computing (HPC) platforms and platforms popularly re- ferred to as NoSQL platforms. Platforms are presented based on their relevance, analysis they support and the data organization model they support.

99 GENERAL AND MISCELLANEOUS↗

Final Report: Multi-State Sharing Initiative

In 2003 a joint effort between the U.S. Department of Homeland Security (DHS) and the U.S. Department of Justice created state and metropolitan intelligence fusion centers. These fusion centers were an effort to share law enforcement, disaster, and terrorism related information and intelligence between state and local jurisdictions and to share terrorism related intelligence between state and local law enforcement agencies and various federal entities. In 2006, DHS commissioned the Oak Ridge National Laboratory to establish and manage a groundbreaking program to assist local, state, and tribal leaders in developing the tools and methods required to anticipate and forestall terrorist events and to enhance disaster response. This program, called the Southeast Region Research Initiative (SERRI), combines science and technology with validated operational approaches to address regionally unique requirements and suggest regional solutions with the potential for national application. In 2009, SERRI sponsored the Multistate Sharing Initiative (MSSI) to assist state and metropolitan intelligence fusion centers with sharing information related to a wider variety of state interests than just terrorism. While these fusion centers have been effective at sharing data across organizations within their respective jurisdictions, their organizational structure makes bilateral communication with federal entities convenient and also allows information to be further disbursed to other local entities when appropriate. The MSSI-developed Suspicious Activity Report (SAR) sharing system allows state-to-state sharing of non-terrorism-related law enforcement and disaster information. Currently, the MSSI SAR system is deployed in Alabama, Kentucky, Tennessee, and South Carolina. About 1 year after implementation, cognizant fusion center personnel from each state were contacted to ascertain the status of their MSSI SAR systems. The overwhelming response from these individuals was that the MSSI SAR system was an outstanding success and contributed greatly to the security and resiliency of their states. At least one state commented that SERRI's implementation of the MSSI SAR actually 'jump started' and accelerated deployment and acceptance of the Nationwide Suspicious Activity Reporting Initiative (NSI). While all states were enthusiastic about their systems, South Carolina and Tennessee appeared to be the heaviest users of their respective systems. With NSI taking the load of sharing SARs with other states, Tennessee has redeployed the MSSI SAR system within Tennessee to allow SAR sharing between state and local organizations including Tennessee's three Homeland Security Regions, eleven Homeland Security Districts, and more than 500 police and sheriff offices, as well as with other states. In one success story from South Carolina, the Economy SAR System was used to compile similar SARs from throughout the state which were then forwarded to field liaison officers, emergency management personnel, and law enforcement officers for action.

99 GENERAL AND MISCELLANEOUS↗

Inferring Group Processes from Computer-Mediated Affective Text Analysis

Political communications in the form of unstructured text convey rich connotative meaning that can reveal underlying group social processes. Previous research has focused on sentiment analysis at the document level, but we extend this analysis to sub-document levels through a detailed analysis of affective relationships between entities extracted from a document. Instead of pure sentiment analysis, which is just positive or negative, we explore nuances of affective meaning in 22 affect categories. Our affect propagation algorithm automatically calculates and displays extracted affective relationships among entities in graphical form in our prototype (TEAMSTER), starting with seed lists of affect terms. Several useful metrics are defined to infer underlying group processes by aggregating affective relationships discovered in a text. Our approach has been validated with annotated documents from the MPQA corpus, achieving a performance gain of 74% over comparable random guessers.

99 GENERAL AND MISCELLANEOUS↗