Show simple item record

dc.contributor.authorSauvanaud, Carla
dc.contributor.authorDholakia, Ajay
dc.contributor.authorGuitart Fernández, Jordi
dc.contributor.authorKim, Chulho
dc.contributor.authorMayes, Peter
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.date.accessioned2020-03-27T14:10:20Z
dc.date.available2021-01-27T01:29:21Z
dc.date.issued2020-01-27
dc.identifier.citationSauvanaud, C. [et al.]. Big data deployment in containerized infrastructures through the interconnection of network namespaces. "Software. Practice and experience", Juliol 2020, vol. 50, núm. 7, p. 1087-1113.
dc.identifier.issn0038-0644
dc.identifier.urihttp://hdl.handle.net/2117/182061
dc.description.abstractBig Data applications tackle the challenge of fast handling of large streams of data. Their performance is not only dependent on the data frameworks implementation and the underlying hardware but also on the deployment scheme and its potential for fast scaling. Consequently, several efforts have focused on the ease of deployment of Big Data applications, notably through the use of containerization. This technology was indeed raised to bring multitenancy and multiprocessing out of clusters, providing high deployment flexibility through lightweight container images. Recent studies have focused mostly on Docker containers. Notwithstanding, this article is actually interested in recent Singularity containers as they provide more security and support high-performance computing (HPC) environments and, in this way, they can make Big Data applications benefit from the specialized hardware of HPC. Singularity 2.x, however, does not isolate network resources as required by most Big Data components. Singularity 3.x allows allocating each container with isolated network resources, but their interconnection requires a nontrivial amount of configuration effort. In this context, this article makes a functional contribution in the form of a deployment scheme based on the interconnection of network namespaces, through underlay and overlay networking approaches, to make Big Data applications easily deployable inside Singularity containers. We provide detailed account of our deployment scheme when using both interconnection approaches in the form of a “how-to-do-it” report, and we evaluate it by comparing three Big Data applications based on Hadoop when performing on a bare-metal infrastructure and on scenarios involving Singularity and Docker instances.
dc.format.extent27 p.
dc.language.isoeng
dc.subjectÀrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
dc.subject.lcshHigh performance computing
dc.subject.lcshBig data
dc.subject.otherHadoop
dc.subject.otherNamespaces
dc.subject.otherSingularity containers
dc.titleBig data deployment in containerized infrastructures through the interconnection of network namespaces
dc.typeArticle
dc.subject.lemacCàlcul intensiu (Informàtica)
dc.subject.lemacMacrodades
dc.contributor.groupUniversitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
dc.identifier.doi10.1002/spe.2793
dc.description.peerreviewedPeer Reviewed
dc.relation.publisherversionhttps://onlinelibrary.wiley.com/doi/epdf/10.1002/spe.2793
dc.rights.accessOpen Access
local.identifier.drac27211295
dc.description.versionPostprint (author's final draft)
dc.relation.projectidinfo:eu-repo/grantAgreement/AGAUR/2017 SGR 1414
dc.relation.projectidinfo:eu-repo/grantAgreement/MINECO/1PE/TIN2015-65316-P
local.citation.authorSauvanaud, C.; Dholakia, A.; Guitart, J.; Kim, C.; Mayes, P.
local.citation.publicationNameSoftware. Practice and experience
local.citation.volume50
local.citation.number7
local.citation.startingPage1087
local.citation.endingPage1113


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder