Engineering topics

Horey, James L

Publications and source records attributed to Horey, James L.

9 records

DOE OSTI2014

Analyzing reliability of virtual machine instances with dynamic pricing in the public cloud

This study presents reliability analysis of virtual machine instances in public cloud environments in the face of dynamic pricing. Different from traditional fixed pricing, dynamic pricing allows price to dynamically fluctuate over arbitrary period of time according to external factors such as supply and demand, excess capacity, etc. This pricing option introduces a new type of fault: virtual machine instances may be unexpectedly terminated due to conflicts in the original bid price and the current offered price. This new class of fault under dynamic pricing may be more dominant than traditional faults in cloud computing environments, where resource availability associated with traditional faults is often above 99.9%. To address and understand this new type of fault, we translated two classic reliability metrics, mean time between failures and availability, to the Amazon Web Services spot market using historical price data. We also validated our findings by submitting actual bids in the spot market. We found that overall, our historical analysis and experimental validation lined up well. Based upon these experimental results, we also provided suggestions and techniques to maximize overall reliability of virtual machine instances under dynamic pricing.

Cloud computing↗

DOE OSTI2013

Performance implications from sizing a VM on multi-core systems: A Data analytic application s view

In this paper, we present a quantitative performance analysis of data analytics applications running on multi-core virtual machines. Such environments form the core of cloud computing. In addition, data analytics applications, such as Cassandra and Hadoop, are becoming increasingly popular on cloud computing platforms. This convergence necessitates a better understanding of the performance and cost implications of such hybrid systems. For example, the very rst step in hosting applications in virtualized environments, requires the user to con gure the number of virtual processors and the size of memory. To understand performance implications of this step, we benchmarked three Yahoo Cloud Serving Benchmark (YCSB) workloads in a virtualized multi-core environment. Our measurements indicate that the performance of Cassandra for YCSB workloads does not heavily depend on the processing capacity of a system, while the size of the data set is critical to performance relative to allocated memory. We also identi ed a strong relationship between the running time of workloads and various hardware events (last level cache loads, misses, and CPU migrations). From this analysis, we provide several suggestions to improve the performance of data analytics applications running on cloud computing environments.

Lim, Seung-Hwan↗

DOE OSTI2013

Enhancing Privacy in Participatory Sensing Applications with Multidimensional Data

Participatory sensing applications rely on individuals to share personal data to produce aggregated models and knowledge. In this setting, privacy concerns can discourage widespread adoption of new applications. We present a privacy-preserving participatory sensing scheme based on negative surveys for both continuous and multivariate categorical data. Without relying on encryption, our algorithms enhance the privacy of sensed data in an energy and computation efficient manner. Simulations and implementation on Android smart phones illustrate how multidimensional data can be aggregated in a useful and privacy-enhancing manner.

Forrest, Stephanie↗

DOE OSTI2012

Enhancing Privacy in Participatory Sensing Applications with Multidimensional Data

Participatory sensing applications rely on individuals to share local and personal data with others to produce aggregated models and knowledge. In this setting, privacy is an important consideration, and lack of privacy could discourage widespread adoption of many exciting applications. We present a privacy-preserving participatory sensing scheme for multidimensional data which uses negative surveys. Multidimensional data, such as vectors of attributes that include location and environment fields, pose a particular challenge for privacy protection and are common in participatory sensing applications. When reporting data in a negative survey, an individual participant randomly selects a value from the set complement of the sensed data value, once for each dimension, and returns the negative values to a central collection server. Using algorithms described in this paper, the server can reconstruct the probability density functions of the original distributions of sensed values, without knowing the participants actual data. As a consequence, complicated encryption and key management schemes are avoided, conserving energy. We study trade-offs between accuracy and privacy, and their relationships to the number of dimensions, categories, and participants. We introduce dimensional adjustment, a method that reduces the magnification of error associated with earlier work. Two simulation scenarios illustrate how the approach can protect the privacy of a participant's multidimensional data while allowing useful population information to be aggregated.

Groat, Michael↗

DOE OSTI2012

Design Principles for Effective Knowledge Discovery from Big Data

Big data phenomenon refers to the practice of collection and processing of very large data sets and associated systems and algorithms used to analyze these massive datasets. Architectures for big data usually range across multiple machines and clusters, and they commonly consist of multiple special purpose sub-systems. Coupled with the knowledge discovery process, big data movement offers many unique opportunities for organizations to benefit (with respect to new insights, business optimizations, etc.). However, due to the difficulty of analyzing such large datasets, big data presents unique systems engineering and architectural challenges. In this paper, we present three sys- tem design principles that can inform organizations on effective analytic and data collection processes, system organization, and data dissemination practices. The principles presented derive from our own research and development experiences with big data problems from various federal agencies, and we illustrate each principle with our own experiences and recommendations.

Architecture↗

DOE OSTI2012

Reconstructing Spatial Distributions from Anonymized Locations

Devices such as mobile phones, tablets, and sensors are often equipped with GPS that accurately report a person's location. Combined with wireless communication, these devices enable a wide range of new social tools and applications. These same qualities, however, leave location-aware applications vulnerable to privacy violations. This paper introduces the Negative Quad Tree, a privacy protection method for location aware applications. The method is broadly applicable to applications that use spatial density information, such as social applications that measure the popularity of social venues. The method employs a simple anonymization algorithm running on mobile devices, and a more complex reconstruction algorithm on a central server. This strategy is well suited to low-powered mobile devices. The paper analyzes the accuracy of the reconstruction method in a variety of simulated and real-world settings and demonstrates that the method is accurate enough to be used in many real-world scenarios.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2012

Big Data Platforms as a Service: Challenges and Approach

Infrastructure-as-a-Service has revolutionized the manner in which users commission computing infrastructure. Coupled with Big Data platforms (Hadoop, Cassandra), IaaS has democratized the ability to store and process massive datasets. For users that need to customize or create new Big Data stacks, however, readily available solutions do not yet exist. Users must first acquire the necessary cloud computing infrastructure, and manually install the prerequisite software. For complex distributed services this can be a daunting challenge. To address this issue, we argue that distributed services should be viewed as a single application consisting of virtual machines. Users should no longer be concerned about individual machines or their internal organization. To illustrate this concept, we introduce Cloud-Get, a distributed package manager that enables the simple installation of distributed services in a cloud computing environment. Cloud-Get enables users to instantiate and modify distributed services, including Big Data services, using simple commands. Cloud-Get also simplifies creating new distributed services via standardized package definitions.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2011

Challenges in Scheduling Aggregation in CyberPhysical Information Processing Systems

Data aggregation (a.k.a reduce operations) is an important element in information processing systems, including MapReduce clusters and cyberphysical networks. Unlike simple sensor networks, all the data in information processing systems must be eventually aggregated. Our goal is to lower overall latency in these systems by intelligently scheduling aggregation on intermediate routing nodes. Unlike previous models, our model explicitly takes into account link latency and computa- tion time. Our model also considers heterogeneous computing capabilities. In order to understand the potential challenges associated with constructing a distributed scheduler that minimizes la- tency, we ve developed a simulation of our model and tested the results of randomly scheduling nodes. Although these experiments were designed to provide data for a null-model, preliminary results have yielded a few interesting observations. We show that in cases where the computation time is larger than transmission time, in-network aggregation can have a large effect (reducing latency by 50% or more), but that naive scheduling can have a detrimental effect. Specifically, we show that when the root node (a.k.a the basestation) is faster than the other nodes, the latency can increase with increased coverage, and that these effects vary with the number of nodes present.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2011

Latency Minimizing Tasking for Information Processing Systems

Real-time cyber-physical systems and information processing clusters require system designers to consider the total latency involved in collecting and aggregating data. For example, applications such as wild-fire monitoring require data to be presented to users in a timely manner. However, most models and algorithms for sensor networks have focused on alternative metrics such as energy efficiency. In this paper, we present a new model of sensor network aggregation that focuses on total latency. Our model is flexible and enables users to configure varying transmission and computation time on a node-by-node basis, and thus enables the simulation of complex computational phenomena. In addition, we present results from three tasking algorithms that trade-off local communication for overall latency performance. These algorithms are evaluated in simulated networks of up to 200 nodes. We've presented an aggregation-focused model of sensor networks that can be used to study the trade-offs between computational coverage and total latency. Our model explicitly takes into account transmission and computation times, and enables users to define different values for the basestation. In addition, we've presented three different tasking algorithms that operate over model to produce aggregation schedules of varying quality. In the future, we expect to continue exploring distributed tasking algorithms for information processing systems. We've shown that the gap between highly optimized schedules that use global information is quite large relative to our distributed algorithms. This gives us encouragement that future distributed tasking algorithms can still make large gains.

99 GENERAL AND MISCELLANEOUS↗