Mostra el registre d'ítem simple
Enabling system wide shared memory for performance improvement in PyCOMPSs applications
dc.contributor.author | Foyer, Clément |
dc.contributor.author | Conejero Bañón, Francisco Javier |
dc.contributor.author | Ejarque Artigas, Jorge |
dc.contributor.author | Badia Sala, Rosa Maria |
dc.contributor.author | Tate, Adrian |
dc.contributor.author | McIntosh-Smith, Simon |
dc.contributor.other | Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors |
dc.contributor.other | Barcelona Supercomputing Center |
dc.date.accessioned | 2021-04-15T13:02:12Z |
dc.date.available | 2021-04-15T13:02:12Z |
dc.date.issued | 2020 |
dc.identifier.citation | Foyer, C. [et al.]. Enabling system wide shared memory for performance improvement in PyCOMPSs applications. A: Workshop on Python for High-Performance and Scientific Computing. "Proceedings of PYHPC 2020, 9th Workshop on Python for High-Performance and Scientific Computing: Held in conjunction with SC20,The International Conference for High Performance Computing, Networking, Storage and Analysis: Virtual Conference, November 9-19, 2020". Institute of Electrical and Electronics Engineers (IEEE), 2020, p. 22-31. ISBN 978-0-7381-1086-8. DOI 10.1109/PyHPC51966.2020.00008. |
dc.identifier.isbn | 978-0-7381-1086-8 |
dc.identifier.uri | http://hdl.handle.net/2117/343794 |
dc.description.abstract | Python has been gaining some traction for years in the world of scientific applications. However, the high-level abstraction it provides may not allow the developer to use the machines to their peak performance. To address this, multiple strategies, sometimes complementary, have been developed to enrich the software ecosystem either by relying on additional libraries dedicated to efficient computation (e.g., NumPy) or by providing a framework to better use HPC scale infrastructures (e.g., PyCOMPSs).In this paper, we present a Python extension based on SharedArray that enables the support of system-provided shared memory and its integration into the PyCOMPSs programming model as an example of integration to a complex Python environment. We also evaluate the impact such a tool may have on performance in two types of distributed execution-flows, one for linear algebra with a blocked matrix multiplication application and the other in the context of data-clustering with a k-means application. We show that with very little modification of the original decorator (3 lines of code to be modified) of the task-based application the gain in performance can rise above 40% for tasks relying heavily on data reuse on a distributed environment, especially when loading the data is prominent in the execution time. |
dc.description.sponsorship | This work was partly funded by the EXPERTISE project (http://www.msca-expertise.eu/), which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 721865. BSC authors have also been supported by the Spanish Government through contracts SEV2015-0493 and TIN2015-65316-P, and by Generalitat de Catalunya through contract 2014-SGR-1051. |
dc.format.extent | 10 p. |
dc.language.iso | eng |
dc.publisher | Institute of Electrical and Electronics Engineers (IEEE) |
dc.subject | Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors |
dc.subject.lcsh | Parallel programming (Computer science) |
dc.subject.lcsh | Parallel processing (Electronic computers) |
dc.subject.lcsh | High-level programming languages |
dc.subject.other | Memory |
dc.subject.other | Shared memory |
dc.subject.other | Task |
dc.subject.other | Python |
dc.subject.other | Distributed memory |
dc.subject.other | NumPy |
dc.subject.other | Data management |
dc.title | Enabling system wide shared memory for performance improvement in PyCOMPSs applications |
dc.type | Conference report |
dc.subject.lemac | Programació en paral·lel (Informàtica) |
dc.subject.lemac | Processament en paral·lel (Ordinadors) |
dc.contributor.group | Universitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions |
dc.identifier.doi | 10.1109/PyHPC51966.2020.00008 |
dc.description.peerreviewed | Peer Reviewed |
dc.relation.publisherversion | https://ieeexplore.ieee.org/document/9307935 |
dc.rights.access | Open Access |
local.identifier.drac | 30574538 |
dc.description.version | Postprint (author's final draft) |
dc.relation.projectid | info:eu-repo/grantAgreement/EC/H2020/721865/EU/models, EXperiments and high PERformance computing for Turbine mechanical Integrity and Structural dynamics in Europe/EXPERTISE |
local.citation.author | Foyer, C.; Conejero, J.; Ejarque, J.; Badia, R.M.; Tate, A.; McIntosh-Smith, S. |
local.citation.contributor | Workshop on Python for High-Performance and Scientific Computing |
local.citation.publicationName | Proceedings of PYHPC 2020, 9th Workshop on Python for High-Performance and Scientific Computing: Held in conjunction with SC20,The International Conference for High Performance Computing, Networking, Storage and Analysis: Virtual Conference, November 9-19, 2020 |
local.citation.startingPage | 22 |
local.citation.endingPage | 31 |