Show simple item record

dc.contributor.authorVillalba Navarro, Álvaro
dc.contributor.authorCarrera Pérez, David
dc.contributor.otherBarcelona Supercomputing Center
dc.contributor.otherUniversitat Politècnica de Catalunya. Doctorat en Arquitectura de Computadors
dc.date.accessioned2019-02-18T15:57:46Z
dc.date.available2019-02-18T15:57:46Z
dc.date.issued2018-12
dc.identifier.citationVillalba, Á.; Carrera, D. Multi-tenant Pub/Sub Processing for Real-Time Data Streams. A: "Euro-Par 2018: Parallel Processing Workshops: Euro-Par 2018 International Workshops, Turin, Italy, August 27-28, 2018: revised selected papers". Springer, 2018, p. 251-526.
dc.identifier.isbn978-3-030-10548-8
dc.identifier.urihttp://hdl.handle.net/2117/129338
dc.description.abstractDevices and sensors generate streams of data across a diversity of locations and protocols. That data usually reaches a central platform that is used to store and process the streams. Processing can be done in real time, with transformations and enrichment happening on-the-fly, but it can also happen after data is stored and organized in repositories. In the former case, stream processing technologies are required to operate on the data; in the latter batch analytics and queries are of common use. This paper introduces a runtime to dynamically construct data stream processing topologies based on user-supplied code. These dynamic topologies are built on-the-fly using a data subscription model defined by the applications that consume data. Each user-defined processing unit is called a Service Object. Every Service Object consumes input data streams and may produce output streams that others can consume. The subscription-based programing model enables multiple users to deploy their own data-processing services. The runtime does the dynamic forwarding of data and execution of Service Objects from different users. Data streams can originate in real-world devices or they can be the outputs of Service Objects. The runtime leverages Apache STORM for parallel data processing, that combined with dynamic user-code injection provides multi-tenant stream processing topologies. In this work we describe the runtime, its features and implementation details, as well as we include a performance evaluation of some of its core components.
dc.description.sponsorshipThis work is partially supported by the European Research Council (ERC) un- der the EU Horizon 2020 programme (GA 639595), the Spanish Ministry of Economy, Industry and Competitivity (TIN2015-65316-P) and the Generalitat de Catalunya (2014-SGR-1051).
dc.format.extent12 p.
dc.language.isoeng
dc.publisherSpringer
dc.subjectÀrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
dc.subject.lcshHigh performance computing
dc.subject.otherBig Data
dc.subject.otherAnalytics
dc.subject.otherStream Processing
dc.subject.otherReal-time Data Processing
dc.subject.otherProgramming Models
dc.subject.otherInternet of Things
dc.titleMulti-tenant Pub/Sub processing for real-time data streams
dc.typeConference lecture
dc.subject.lemacSupercomputadors
dc.identifier.doi10.1007/978-3-030-10549-5_20
dc.description.peerreviewedPeer Reviewed
dc.relation.publisherversionhttps://link.springer.com/chapter/10.1007/978-3-030-10549-5_20
dc.rights.accessOpen Access
local.identifier.drac28700556
dc.description.versionPostprint (author's final draft)
dc.relation.projectidinfo:eu-repo/grantAgreement/EC/H2020/639595/EU/Holistic Integration of Emerging Supercomputing Technologies/Hi-EST
dc.relation.projectidinfo:eu-repo/grantAgreement/MINECO//TIN2015-65316-P/ES/COMPUTACION DE ALTAS PRESTACIONES VII/
dc.relation.projectidinfo:eu-repo/grantAgreement/AGAUR/V PRI/2014 SGR 1051
local.citation.publicationNameEuro-Par 2018: Parallel Processing Workshops: Euro-Par 2018 International Workshops, Turin, Italy, August 27-28, 2018: revised selected papers
local.citation.volume11339
local.citation.startingPage251
local.citation.endingPage526


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record