Show simple item record

dc.contributor.authorSauvanaud, Carla
dc.contributor.authorDholakia, Ajay
dc.contributor.authorGuitart Fernández, Jordi
dc.contributor.authorKim, Chulho
dc.contributor.authorMayes, Peter
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.identifier.citationSauvanaud, C. [et al.]. Big data deployment in containerized infrastructures through the interconnection of network namespaces. "Software. Practice and experience", Juliol 2020, vol. 50, núm. 7, p. 1087-1113.
dc.description.abstractBig Data applications tackle the challenge of fast handling of large streams of data. Their performance is not only dependent on the data frameworks implementation and the underlying hardware but also on the deployment scheme and its potential for fast scaling. Consequently, several efforts have focused on the ease of deployment of Big Data applications, notably through the use of containerization. This technology was indeed raised to bring multitenancy and multiprocessing out of clusters, providing high deployment flexibility through lightweight container images. Recent studies have focused mostly on Docker containers. Notwithstanding, this article is actually interested in recent Singularity containers as they provide more security and support high-performance computing (HPC) environments and, in this way, they can make Big Data applications benefit from the specialized hardware of HPC. Singularity 2.x, however, does not isolate network resources as required by most Big Data components. Singularity 3.x allows allocating each container with isolated network resources, but their interconnection requires a nontrivial amount of configuration effort. In this context, this article makes a functional contribution in the form of a deployment scheme based on the interconnection of network namespaces, through underlay and overlay networking approaches, to make Big Data applications easily deployable inside Singularity containers. We provide detailed account of our deployment scheme when using both interconnection approaches in the form of a “how-to-do-it” report, and we evaluate it by comparing three Big Data applications based on Hadoop when performing on a bare-metal infrastructure and on scenarios involving Singularity and Docker instances.
dc.format.extent27 p.
dc.subjectÀrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
dc.subject.lcshHigh performance computing
dc.subject.lcshBig data
dc.subject.otherSingularity containers
dc.titleBig data deployment in containerized infrastructures through the interconnection of network namespaces
dc.subject.lemacCàlcul intensiu (Informàtica)
dc.contributor.groupUniversitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
dc.description.peerreviewedPeer Reviewed
dc.rights.accessOpen Access
dc.description.versionPostprint (author's final draft)
dc.relation.projectidinfo:eu-repo/grantAgreement/AGAUR/2017 SGR 1414
local.citation.authorSauvanaud, C.; Dholakia, A.; Guitart, J.; Kim, C.; Mayes, P.
local.citation.publicationNameSoftware. Practice and experience

Files in this item


This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder