Empowering automatic data-center management with machine learning

Abstract

The Cloud as computing paradigm has become nowadays crucial for most Internet business models. Managing a cloud and optimizing its performance on a moment-by-moment basis is not easy given as the amount and diversity of elements involved (hardware, applications, workloads, customer needsdots). Here we show how a combination of scheduling algorithms and data mining techniques helps improving the performance and profitability of a data-center running virtualized web-services. We model the data-center's main resources (CPU, memory, I/O), quality of service (viewed as response time), and workloads (incoming streams of requests) from past executions. We show how these models to help scheduling algorithms make better decisions about job and resource allocation, aiming for a balance between throughput, quality of service, and power consumption. We test our approach first with real data and web-services on a data-center simulator, and further validate it in a real execution on a reduced scale cluster running the Open-Nebula virtualization platform.