Preemption-aware planning on big-data systems
- Resource Type
- Authors
- Matteo Mazzucchelli; Roberto Cordone; Marco Rabozzi; Marco D. Santambrogio; Giovanni Matteo Fumarola
- Source
- PPOPP
- Subject
- 020203 distributed computing
Computer science
business.industry
Distributed computing
Big data
0202 electrical engineering, electronic engineering, information engineering
Preemption
020207 software engineering
02 engineering and technology
business
Computer Graphics and Computer-Aided Design
Software
Scheduling (computing)
- Language
Recent developments in Big Data frameworks are moving towards reservation based approaches as a mean to manage the increasingly complex mix of computations, whereas preemption techniques are employed to meet strict jobs deadlines. Within this work we propose and evaluate a new planning algorithm in the context of reservation based scheduling. Our approach is able to achieve high cluster utilization while minimizing the need for preemption that causes system overheads and planning mispredictions.