Show simple item record

dc.contributor.authorVillalba Navarro, Álvaro
dc.contributor.authorBerral García, Josep Lluís
dc.contributor.authorCarrera Pérez, David
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.date.accessioned2018-10-04T07:44:10Z
dc.date.available2018-10-04T07:44:10Z
dc.date.issued2019-03
dc.identifier.citationVillalba, Á., Berral, J., Carrera, D. Constant-time sliding window framework with reduced memory footprint and efficient bulk evictions. "IEEE transactions on parallel and distributed systems", Març 2019, vol. 30, núm. 3, p. 486-500.
dc.identifier.issn1045-9219
dc.identifier.urihttp://hdl.handle.net/2117/121867
dc.description.abstractThe fast evolution of data analytics platforms has resulted in an increasing demand for real-time data stream processing. From Internet of Things applications to the monitoring of telemetry generated in large data centers, a common demand for currently emerging scenarios is the need to process vast amounts of data with low latencies, generally performing the analysis process as close to the data source as possible. Stream processing platforms are required to be malleable and absorb spikes generated by fluctuations of data generation rates. Data is usually produced as time series that have to be aggregated using multiple operators, being sliding windows one of the most common abstractions used to process data in real-time. To satisfy the above-mentioned demands, efficient stream processing techniques that aggregate data with minimal computational cost need to be developed. In this paper we present the Monoid Tree Aggregator general sliding window aggregation framework, which seamlessly combines the following features: amortized O(1) time complexity and a worst-case of O(log n) between insertions; it provides both a window aggregation mechanism and a window slide policy that are user programmable; the enforcement of the window sliding policy exhibits amortized O(1) computational cost for single evictions and supports bulk evictions with cost O(log n) ; and it requires a local memory space of O(log n) . The framework can compute aggregations over multiple data dimensions, and has been designed to support decoupling computation and data storage through the use of distributed Key-Value Stores to keep window elements and partial aggregations.
dc.format.extent14 p.
dc.language.isoeng
dc.subjectÀrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
dc.subject.lcshInternet of things
dc.subject.lcshBig data
dc.subject.lcshData mining
dc.subject.otherData analytics
dc.subject.otherStream processing
dc.subject.otherReal-time
dc.titleConstant-time sliding window framework with reduced memory footprint and efficient bulk evictions
dc.typeArticle
dc.subject.lemacInternet de les coses
dc.subject.lemacMacrodades
dc.subject.lemacMineria de dades
dc.contributor.groupUniversitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
dc.identifier.doi10.1109/TPDS.2018.2868960
dc.description.peerreviewedPeer Reviewed
dc.relation.publisherversionhttps://ieeexplore.ieee.org/document/8456588
dc.rights.accessOpen Access
drac.iddocument23357868
dc.description.versionPostprint (published version)
dc.relation.projectidinfo:eu-repo/grantAgreement/EC/H2020/639595
dc.relation.projectidinfo:eu-repo/grantAgreement/MINECO/1PE/TIN2015-65316-P
dc.relation.projectidinfo:eu-repo/grantAgreement/AGAUR/VPRI/2014 SGR 1051
upcommons.citation.authorVillalba, Á., Berral, J., Carrera, D.
upcommons.citation.publishedtrue
upcommons.citation.publicationNameIEEE transactions on parallel and distributed systems
upcommons.citation.volume30
upcommons.citation.number3
upcommons.citation.startingPage486
upcommons.citation.endingPage500


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder