Show simple item record

dc.contributor.authorPoggi, Nicolas
dc.contributor.authorBerral García, Josep Lluís
dc.contributor.authorFenech, Thomas
dc.contributor.authorCarrera Pérez, David
dc.contributor.authorBlakeley, Jose
dc.contributor.authorMinhas, Umar F.
dc.contributor.authorVujic, Nikola
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.identifier.citationPoggi, N., Berral, J., Fenech, T., Carrera, D., Blakeley, J., Minhas, U., Vujic, N. The state of SQL-on-Hadoop in the cloud. A: IEEE International Conference on Big Data. "2016 IEEE International Conference on Big Data: Dec 05-Dec 08, 2015, Washington D.C., USA: proceedings". Institute of Electrical and Electronics Engineers (IEEE), 2016, p. 1432-1443.
dc.description.abstractManaged Hadoop in the cloud, especially SQL-on-Hadoop, has been gaining attention recently. On Platform-as-a-Service (PaaS), analytical services like Hive and Spark come preconfigured for general-purpose and ready to use. Thus, giving companies a quick entry and on-demand deployment of ready SQL-like solutions for their big data needs. This study evaluates cloud services from an end-user perspective, comparing providers including: Microsoft Azure, Amazon Web Services, Google Cloud, and Rackspace. The study focuses on performance, readiness, scalability, and cost-effectiveness of the different solutions at entry/test level clusters sizes. Results are based on over 15,000 Hive queries derived from the industry standard TPC-H benchmark. The study is framed within the ALOJA research project, which features an open source benchmarking and analysis platform that has been recently extended to support SQL-on-Hadoop engines. The ALOJA Project aims to lower the total cost of ownership (TCO) of big data deployments and study their performance characteristics for optimization. The study benchmarks cloud providers across a diverse range instance types, and uses input data scales from 1GB to 1TB, in order to survey the popular entry-level PaaS SQL-on-Hadoop solutions, thereby establishing a common results-base upon which subsequent research can be carried out by the project. Initial results already show the main performance trends to both hardware and software configuration, pricing, similarities and architectural differences of the evaluated PaaS solutions. Whereas some providers focus on decoupling storage and computing resources while offering network-based elastic storage, others choose to keep the local processing model from Hadoop for high performance, but reducing flexibility. Results also show the importance of application-level tuning and how keeping up-to-date hardware and software stacks can influence performance even more than replicating the on-premises model in the cloud.
dc.description.sponsorshipThis work is partially supported by the Microsoft Azure for Research program, the European Research Council (ERC) under the EUs Horizon 2020 programme (GA 639595), the Spanish Ministry of Education (TIN2015-65316-P), and the Generalitat de Catalunya (2014-SGR-1051).
dc.format.extent12 p.
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)
dc.subjectÀrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
dc.subject.lcshBig data
dc.subject.lcshCloud computing
dc.subject.otherManaged Haddop
dc.subject.otherPlatform-as-a-Service (PaaS)
dc.titleThe state of SQL-on-Hadoop in the cloud
dc.typeConference report
dc.subject.lemacComputació en núvol
dc.contributor.groupUniversitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
dc.description.peerreviewedPeer Reviewed
dc.rights.accessOpen Access
dc.description.versionPostprint (author's final draft)
dc.relation.projectidinfo:eu-repo/grantAgreement/EC/H2020/639595/EU/Holistic Integration of Emerging Supercomputing Technologies/Hi-EST
upcommons.citation.authorPoggi, N.; Berral, J.; Fenech, T.; Carrera, D.; Blakeley, J.; Minhas, U.; Vujic, N.
upcommons.citation.contributorIEEE International Conference on Big Data
upcommons.citation.publicationName2016 IEEE International Conference on Big Data: Dec 05-Dec 08, 2015, Washington D.C., USA: proceedings

Files in this item


This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder