Mostra el registre d'ítem simple
Upgrading a high performance computing environment for massive data processing
dc.contributor.author | Ponce, Lucas M. |
dc.contributor.author | dos Santos, Walter |
dc.contributor.author | Meira Jr, Wagner |
dc.contributor.author | Guedes, Dorgival |
dc.contributor.author | Lezzi, Daniele |
dc.contributor.author | Badia Sala, Rosa Maria |
dc.contributor.other | Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors |
dc.contributor.other | Barcelona Supercomputing Center |
dc.date.accessioned | 2020-05-08T09:10:15Z |
dc.date.available | 2020-05-08T09:10:15Z |
dc.date.issued | 2019-10-16 |
dc.identifier.citation | Ponce, L. [et al.]. Upgrading a high performance computing environment for massive data processing. "Journal of Internet services and applications", 16 Octubre 2019, vol. 10, article 19, p. 1-18. |
dc.identifier.issn | 1867-4828 |
dc.identifier.uri | http://hdl.handle.net/2117/186788 |
dc.description.abstract | High-performance computing (HPC) and massive data processing (Big Data) are two trends that are beginning to converge. In that process, aspects of hardware architectures, systems support and programming paradigms are being revisited from both perspectives. This paper presents our experience on this path of convergence with the proposal of a framework that addresses some of the programming issues derived from such integration. Our contribution is the development of an integrated environment that integretes (i) COMPSs, a programming framework for the development and execution of parallel applications for distributed infrastructures; (ii) Lemonade, a data mining and analysis tool; and (iii) HDFS, the most widely used distributed file system for Big Data systems. To validate our framework, we used Lemonade to create COMPSs applications that access data through HDFS, and compared them with equivalent applications built with Spark, a popular Big Data framework. The results show that the HDFS integration benefits COMPSs by simplifying data access and by rearranging data transfer, reducing execution time. The integration with Lemonade facilitates COMPSs’s use and may help its popularization in the Data Science community, by providing efficient algorithm implementations for experts from the data domain that want to develop applications with a higher level abstraction. |
dc.description.sponsorship | Funding This work was partially funded by Fapemig, CAPES, CNPq, MCT/CNPq-InWeb, FAPEMIG-PRONEX-MASWeb (APQ-01400-14), and by the collaboration between Brazilian MCT/RNP and the European Union Horizon 2020 research and innovation programme under grants 690116 (EUBra-BIGSEA) and 777154 (EUBra Atmosphere). |
dc.format.extent | 18 p. |
dc.language.iso | eng |
dc.publisher | Springer |
dc.rights | Attribution 4.0 International |
dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ |
dc.subject | Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors |
dc.subject.lcsh | Big data |
dc.subject.lcsh | High performance computing |
dc.subject.other | COMPSs |
dc.subject.other | HDFS |
dc.subject.other | Lemonade |
dc.title | Upgrading a high performance computing environment for massive data processing |
dc.type | Article |
dc.subject.lemac | Macrodades |
dc.subject.lemac | Càlcul intensiu (Informàtica) |
dc.contributor.group | Universitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions |
dc.identifier.doi | 10.1186/s13174-019-0118-7 |
dc.description.peerreviewed | Peer Reviewed |
dc.relation.publisherversion | https://link.springer.com/article/10.1186/s13174-019-0118-7 |
dc.rights.access | Open Access |
local.identifier.drac | 28098368 |
dc.description.version | Postprint (published version) |
dc.relation.projectid | info:eu-repo/grantAgreement/EC/H2020/690116/EU/EUrope-BRAzil Collaboration on BIG Data Scientific REsearch through Cloud-Centric Applications/EUBra-BIGSEA |
local.citation.author | Ponce, L.; dos Santos, W.; Meira Jr, W.; Guedes, D.; Lezzi, D.; Badia, R.M. |
local.citation.publicationName | Journal of Internet services and applications |
local.citation.volume | 10 |
local.citation.number | article 19 |
local.citation.startingPage | 1 |
local.citation.endingPage | 18 |
Fitxers d'aquest items
Aquest ítem apareix a les col·leccions següents
-
Articles de revista [317]
-
Articles de revista [1.046]
-
Articles de revista [382]