Phase level energy aware map reduce scheduling for big data applications
- Resource Type
- Conference
- Authors
- Shah, Manisha; Shukla, Piyush Kumar; Pandey, Rajeev
- Source
- 2016 International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES) Signal Processing, Communication, Power and Embedded System (SCOPES), 2016 International Conference on. :532-535 Oct, 2016
- Subject
- Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Computing and Processing
Engineered Materials, Dielectrics and Plasmas
Engineering Profession
Fields, Waves and Electromagnetics
General Topics for Engineers
Photonics and Electrooptics
Power, Energy and Industry Applications
Robotics and Control Systems
Signal Processing and Analysis
Energy consumption
Signal processing algorithms
Scheduling algorithms
Clustering algorithms
Energy efficiency
Scheduling
mapreduce
scheduling
hadoop
bigdata
workload
- Language
The preponderance of large scale data radical applications executed by various business areas for performing data preparation and data analytics based on Map Reduce paradigm which can be better implemented on Hadoop. Such data driven applications which are executed on large clusters set up in data centers hike the energy cost which imposes burden on overall data center cost. Thus minimizing parameter that guides energy consumption becomes paramount requisite to be considered. In this paper we propose a framework for improving energy efficiency of Map Reduce applications. We propose phase level energy aware map reduce scheduling algorithms that assign map and reduce task to system on the basis of maximum node availability. We perform various extensive experiments on Hadoop cluster to determine execution time and energy consumption for several workloads from Hadoop including Terasort and K-means clustering and results evaluated that proposed algorithm consume less energy than various heuristic algorithms and minimizes execution time.