PTC: Performance Tuning Component for Auto-Tuning of MapReduce's Configuration

Prasad, Ramesh (2017) PTC: Performance Tuning Component for Auto-Tuning of MapReduce's Configuration. Masters thesis, Dublin, National College of Ireland.

Preview

PDF (Master of Science) Download (2MB) | Preview

Abstract

Hadoop's MapReduce framework was developed to process large datasets in a distributed environment. Performance of MapReduce job is driven by large number of settings and configuration parameters. Manual configuration of these parameters and identification of optimal values is an error prone and tedious task. Improving Performance of MapReduce framework is important in order to effectively utilize the resource. In this work, existing research methodologies has been evaluated to understand the impact of these configuration parameters and approaches to identify their optimal values. In this research, we propose Performance Tuning Component for Auto-Tuning of configuration parameter with optimal values of io.sort.factor and mapreduce.job.reduces parameters. Prediction model has been developed in order to find the optimal values of these parameters using Ridge Regression Algorithm. Prediction model has accuracy of 91.26% for predicting values of configuration parameters. TeraSort and WordCount have been used as benchmark for evaluation of performance for MapReduce job. Experiments results show that the proposed solution provided about 6-48% improvement in execution time for different MapReduce jobs executed. Cost of MapReduce in cloud is associated with the time spent and resources utilized by MapReduce job. Hence, our proposed solution will not only save time, but also cost associated by reducing the execution time of the MapReduce job.

Item Type:	Thesis (Masters)
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Cloud computing
Divisions:	School of Computing > Master of Science in Cloud Computing
Depositing User:	Caoimhe Ní Mhaicín
Date Deposited:	21 Nov 2017 14:32
Last Modified:	21 Nov 2017 14:32
URI:	https://norma.ncirl.ie/id/eprint/2876

Actions (login required)

View Item