Show simple item record

dc.contributor.authorBerral García, Josep Lluís
dc.contributor.authorPoggi Mastrokalo, Nicolas
dc.contributor.authorCarrera Pérez, David
dc.contributor.authorCall, Aaron
dc.contributor.authorReinauer, Rob
dc.contributor.authorGreen, Daron
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.contributor.otherBarcelona Supercomputing Center
dc.date.accessioned2017-05-26T10:44:17Z
dc.date.available2017-05-26T10:44:17Z
dc.date.issued2015-10
dc.identifier.citationBerral, J., Poggi, N., Carrera, D., Call, A., Reinauer, R., Green, D. ALOJA: A framework for benchmarking and predictive analytics in Hadoop deployments. "IEEE Transactions on emerging topics in computing", Oct-Des 2017, vol. 5, núm. 4, p. 480-493.
dc.identifier.issn2168-6750
dc.identifier.urihttp://hdl.handle.net/2117/104910
dc.description.abstractThis article presents the ALOJA project and its analytics tools, which leverages machine learning to interpret Big Data benchmark performance data and tuning. ALOJA is part of a long-term collaboration between BSC and Microsoft to automate the characterization of cost-effectiveness on Big Data deployments, currently focusing on Hadoop. Hadoop presents a complex run-time environment, where costs and performance depend on a large number of configuration choices. The ALOJA project has created an open, vendor-neutral repository, featuring over 40,000 Hadoop job executions and their performance details. The repository is accompanied by a test-bed and tools to deploy and evaluate the cost-effectiveness of different hardware configurations, parameters and Cloud services. Despite early success within ALOJA, a comprehensive study requires automation of modeling procedures to allow an analysis of large and resource-constrained search spaces. The predictive analytics extension, ALOJA-ML, provides an automated system allowing knowledge discovery by modeling environments from observed executions. The resulting models can forecast execution behaviors, predicting execution times for new configurations and hardware choices. That also enables model-based anomaly detection or efficient benchmark guidance by prioritizing executions. In addition, the community can benefit from ALOJA data-sets and framework to improve the design and deployment of Big Data applications.
dc.description.sponsorshipThis project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 639595). This work is partially supported by the Ministry of Economy of Spain under contracts TIN2012-34557 and 2014SGR1051.
dc.format.extent14 p.
dc.language.isoeng
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)
dc.rightsAttribution 3.0 Spain
dc.rights.urihttp://creativecommons.org/licenses/by/3.0/es/
dc.subjectÀrees temàtiques de la UPC::Informàtica::Intel·ligència artificial
dc.subject.lcshMachine learning
dc.subject.lcshBig data
dc.subject.otherData-center management
dc.subject.otherHadoop
dc.subject.otherBenchmarks
dc.subject.otherModeling and prediction
dc.subject.otherExecution experiences
dc.titleALOJA: A framework for benchmarking and predictive analytics in Hadoop deployments
dc.typeArticle
dc.subject.lemacAprenentatge automàtic
dc.subject.lemacMacrodades
dc.contributor.groupUniversitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
dc.identifier.doi10.1109/TETC.2015.2496504
dc.description.peerreviewedPeer Reviewed
dc.relation.publisherversionhttp://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7312954
dc.rights.accessOpen Access
local.identifier.drac17385073
dc.description.versionPostprint (published version)
dc.relation.projectidinfo:eu-repo/grantAgreement/MINECO/6PN/TIN2012-34557
dc.relation.projectidinfo:eu-repo/grantAgreement/EC/H2020/639595/EU/Holistic Integration of Emerging Supercomputing Technologies/Hi-EST
local.citation.authorBerral, J.; Poggi, N.; Carrera, D.; Call, A.; Reinauer, R.; Green, D.
local.citation.publicationNameIEEE Transactions on emerging topics in computing
local.citation.startingPage480
local.citation.endingPage493


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record

Attribution 3.0 Spain
Except where otherwise noted, content on this work is licensed under a Creative Commons license : Attribution 3.0 Spain