ALOJA: A benchmarking and predictive platform for big data performance analysis
Document typeConference report
Rights accessOpen Access
The main goals of the ALOJA research project from BSC-MSR, are to explore and automate the characterization of cost-effectivenessof Big Data deployments. The development of the project over its first year, has resulted in a open source benchmarking platform, an online public repository of results with over 42,000 Hadoop job runs, and web-based analytic tools to gather insights about system's cost-performance1. This article describes the evolution of the project's focus and research lines from over a year of continuously benchmarking Hadoop under dif- ferent configuration and deployments options, presents results, and dis cusses the motivation both technical and market-based of such changes. During this time, ALOJA's target has evolved from a previous low-level profiling of Hadoop runtime, passing through extensive benchmarking and evaluation of a large body of results via aggregation, to currently leveraging Predictive Analytics (PA) techniques. Modeling benchmark executions allow us to estimate the results of new or untested configu- rations or hardware set-ups automatically, by learning techniques from past observations saving in benchmarking time and costs.
CitationPoggi, N., Berral, J., Carrera, D. ALOJA: A benchmarking and predictive platform for big data performance analysis. A: International Workshop on Big Data Benchmarking. "Big Data Benchmarking: 6th International Workshop, WBDB 2015: Toronto, ON, Canada, June 16-17, 2015 and 7th International Workshop, WBDB 2015: New Delhi, India, December 14-15, 2015: revised selected papers". Toronto: Springer, 2016, p. 71-84.