Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form
D-Scholarship @ Pitt Banner and Links to Homepage

Cost-Effective Resource Provisioning for MapReduce in a Cloud

Palanisamy, B and Singh, A and Liu, L (2015) Cost-Effective Resource Provisioning for MapReduce in a Cloud. IEEE Transactions on Parallel and Distributed Systems, 26 (5). 1265 - 1279. ISSN 1045-9219

[img]
Preview
 PDF
Available under License : See the attached license file.
Download (6MB) | Preview
[img] Plain Text (licence)
Available under License : See the attached license file.
Download (1kB)

Abstract

This paper presents a new MapReduce cloud service model, Cura, for provisioning cost-effective MapReduce services in a cloud. In contrast to existing MapReduce cloud services such as a generic compute cloud or a dedicated MapReduce cloud, Cura has a number of unique benefits. First, Cura is designed to provide a cost-effective solution to efficiently handle MapReduce production workloads that have a significant amount of interactive jobs. Second, unlike existing services that require customers to decide the resources to be used for the jobs, Cura leverages MapReduce profiling to automatically create the best cluster configuration for the jobs. While the existing models allow only a per-job resource optimization for the jobs, Cura implements a globally efficient resource allocation scheme that significantly reduces the resource usage cost in the cloud. Third, Cura leverages unique optimization opportunities when dealing with workloads that can withstand some slack. By effectively multiplexing the available cloud resources among the jobs based on the job requirements, Cura achieves significantly lower resource usage costs for the jobs. Cura's core resource management schemes include cost-aware resource provisioning, VM-aware scheduling and online virtual machine reconfiguration. Our experimental results using Facebook-like workload traces show that our techniques lead to more than 80 percent reduction in the cloud compute infrastructure cost with upto 65 percent reduction in job response times.

Share

Citation/Export:
Social Networking:
Share |

Details

Item Type: Article
Status: Published
Creators/Authors:
CreatorsEmailPitt UsernameORCID
Palanisamy, BBPALAN@pitt.eduBPALAN
Singh, A
Liu, L
Date: 1 May 2015
Date Type: Publication
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Journal or Publication Title: IEEE Transactions on Parallel and Distributed Systems
Volume: 26
Number: 5
Page Range: 1265 - 1279
DOI or Unique Handle: 10.1109/tpds.2014.2320498
Institution: University of Pittsburgh
Schools and Programs: School of Information Sciences > Information Science
Refereed: Yes
ISSN: 1045-9219
Date Deposited: 24 Jun 2014 15:43
Last Modified: 30 Mar 2021 13:55
URI: http://d-scholarship.pitt.edu/id/eprint/22052

Metrics

Monthly Views for the past 3 years

Plum Analytics

Altmetric.com

Actions (login required)

View Item View Item