eCite Digital Repository

Cost efficient scheduling of MapReduce applications on public clouds


Zeng, X and Garg, SK and Wen, Z and Strazdins, P and Zomaya, AY and Ranjan, R, Cost efficient scheduling of MapReduce applications on public clouds, Journal of Computational Science, 26 pp. 375-388. ISSN 1877-7503 (2018) [Refereed Article]

PDF (Accepted manuscript)

Copyright Statement

2017 Elsevier B.V.

DOI: doi:10.1016/j.jocs.2017.07.017


MapReduce framework has been one of the most prominent ways for efficient processing large amount of data requiring huge computational capacity. On-demand computing resources of Public Clouds have become a natural host for these MapReduce applications. However, the decision of what type and in what amount computing and storage resources should be rented is still a userís responsibility. This is not a trivial task particularly when users may have performance constraints such as deadline and have several Cloud product types to choose with the intention of not spending much money. Even though there are several existing scheduling systems, however, most of them are not developed to manage the scheduling of MapReduce applications. That is, they do not consider things such as number of map and reduce tasks that are needed to be scheduled and heterogeneity of Virtual Machines (VMs) available. This paper proposes a novel greedy-based MapReduce application scheduling algorithm (MASA) that considers the userís constraints in order to minimize cost of renting Cloud resources while considering Service Level Agreements (SLA) in terms of the user given budget and deadline constraints. The simulation results show that MASA can achieve 25-50% cost reduction in comparison to current SLA agnostic methods and there is only 10% performance disparity between MASA and an exhaustive search algorithm.

Item Details

Item Type:Refereed Article
Keywords:cloud computing, big data, map reduce, service level agreement, scheduling, cross layer
Research Division:Information and Computing Sciences
Research Group:Distributed computing and systems software
Research Field:Distributed systems and algorithms
Objective Division:Information and Communication Services
Objective Group:Information systems, technologies and services
Objective Field:Information systems, technologies and services not elsewhere classified
UTAS Author:Garg, SK (Dr Saurabh Garg)
ID Code:120235
Year Published:2018 (online first 2017)
Web of Science® Times Cited:13
Deposited By:Information and Communication Technology
Deposited On:2017-08-17
Last Modified:2018-09-10
Downloads:71 View Download Statistics

Repository Staff Only: item control page