Mostra el registre d'ítem simple

dc.contributor.authorPonce, Lucas M.
dc.contributor.authordos Santos, Walter
dc.contributor.authorMeira Jr, Wagner
dc.contributor.authorGuedes, Dorgival
dc.contributor.authorLezzi, Daniele
dc.contributor.authorBadia Sala, Rosa Maria
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.contributor.otherBarcelona Supercomputing Center
dc.date.accessioned2020-05-08T09:10:15Z
dc.date.available2020-05-08T09:10:15Z
dc.date.issued2019-10-16
dc.identifier.citationPonce, L. [et al.]. Upgrading a high performance computing environment for massive data processing. "Journal of Internet services and applications", 16 Octubre 2019, vol. 10, article 19, p. 1-18.
dc.identifier.issn1867-4828
dc.identifier.urihttp://hdl.handle.net/2117/186788
dc.description.abstractHigh-performance computing (HPC) and massive data processing (Big Data) are two trends that are beginning to converge. In that process, aspects of hardware architectures, systems support and programming paradigms are being revisited from both perspectives. This paper presents our experience on this path of convergence with the proposal of a framework that addresses some of the programming issues derived from such integration. Our contribution is the development of an integrated environment that integretes (i) COMPSs, a programming framework for the development and execution of parallel applications for distributed infrastructures; (ii) Lemonade, a data mining and analysis tool; and (iii) HDFS, the most widely used distributed file system for Big Data systems. To validate our framework, we used Lemonade to create COMPSs applications that access data through HDFS, and compared them with equivalent applications built with Spark, a popular Big Data framework. The results show that the HDFS integration benefits COMPSs by simplifying data access and by rearranging data transfer, reducing execution time. The integration with Lemonade facilitates COMPSs’s use and may help its popularization in the Data Science community, by providing efficient algorithm implementations for experts from the data domain that want to develop applications with a higher level abstraction.
dc.description.sponsorshipFunding This work was partially funded by Fapemig, CAPES, CNPq, MCT/CNPq-InWeb, FAPEMIG-PRONEX-MASWeb (APQ-01400-14), and by the collaboration between Brazilian MCT/RNP and the European Union Horizon 2020 research and innovation programme under grants 690116 (EUBra-BIGSEA) and 777154 (EUBra Atmosphere).
dc.format.extent18 p.
dc.language.isoeng
dc.publisherSpringer
dc.rightsAttribution 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectÀrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
dc.subject.lcshBig data
dc.subject.lcshHigh performance computing
dc.subject.otherCOMPSs
dc.subject.otherHDFS
dc.subject.otherLemonade
dc.titleUpgrading a high performance computing environment for massive data processing
dc.typeArticle
dc.subject.lemacMacrodades
dc.subject.lemacCàlcul intensiu (Informàtica)
dc.contributor.groupUniversitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
dc.identifier.doi10.1186/s13174-019-0118-7
dc.description.peerreviewedPeer Reviewed
dc.relation.publisherversionhttps://link.springer.com/article/10.1186/s13174-019-0118-7
dc.rights.accessOpen Access
local.identifier.drac28098368
dc.description.versionPostprint (published version)
dc.relation.projectidinfo:eu-repo/grantAgreement/EC/H2020/690116/EU/EUrope-BRAzil Collaboration on BIG Data Scientific REsearch through Cloud-Centric Applications/EUBra-BIGSEA
local.citation.authorPonce, L.; dos Santos, W.; Meira Jr, W.; Guedes, D.; Lezzi, D.; Badia, R.M.
local.citation.publicationNameJournal of Internet services and applications
local.citation.volume10
local.citation.numberarticle 19
local.citation.startingPage1
local.citation.endingPage18


Fitxers d'aquest items

Thumbnail

Aquest ítem apareix a les col·leccions següents

Mostra el registre d'ítem simple