Show simple item record

dc.contributor.authorVillalba Navarro, Álvaro
dc.contributor.authorBerral García, Josep Lluís
dc.contributor.authorCarrera Pérez, David
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.contributor.otherBarcelona Supercomputing Center
dc.contributor.otherUniversitat Politècnica de Catalunya. Doctorat en Arquitectura de Computadors
dc.identifier.citationVillalba, Á., Berral, J., Carrera, D. Constant-time sliding window framework with reduced memory footprint and efficient bulk evictions. "IEEE transactions on parallel and distributed systems", Març 2019, vol. 30, núm. 3, p. 486-500.
dc.description.abstractThe fast evolution of data analytics platforms has resulted in an increasing demand for real-time data stream processing. From Internet of Things applications to the monitoring of telemetry generated in large data centers, a common demand for currently emerging scenarios is the need to process vast amounts of data with low latencies, generally performing the analysis process as close to the data source as possible. Stream processing platforms are required to be malleable and absorb spikes generated by fluctuations of data generation rates. Data is usually produced as time series that have to be aggregated using multiple operators, being sliding windows one of the most common abstractions used to process data in real-time. To satisfy the above-mentioned demands, efficient stream processing techniques that aggregate data with minimal computational cost need to be developed. In this paper we present the Monoid Tree Aggregator general sliding window aggregation framework, which seamlessly combines the following features: amortized O(1) time complexity and a worst-case of O(log n) between insertions; it provides both a window aggregation mechanism and a window slide policy that are user programmable; the enforcement of the window sliding policy exhibits amortized O(1) computational cost for single evictions and supports bulk evictions with cost O(log n) ; and it requires a local memory space of O(log n) . The framework can compute aggregations over multiple data dimensions, and has been designed to support decoupling computation and data storage through the use of distributed Key-Value Stores to keep window elements and partial aggregations.
dc.description.sponsorshipThis project is partially supported by the European Research Council (ERC), Spain under the European Unions Horizon 2020 research and innovation programme (grant agreement No 639595). It is also partially supported by the Ministry of Economy of Spain under contract TIN2015- 65316-P and Generalitat de Catalunya, Spain under contract 2014SGR1051, by the ICREA Academia program, and by the BSC-CNS Severo Ochoa program (SEV-2015-0493).
dc.format.extent14 p.
dc.subjectÀrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
dc.subject.lcshInternet of things
dc.subject.lcshBig data
dc.subject.lcshData mining
dc.subject.otherData analytics
dc.subject.otherStream processing
dc.titleConstant-time sliding window framework with reduced memory footprint and efficient bulk evictions
dc.subject.lemacInternet de les coses
dc.subject.lemacMineria de dades
dc.contributor.groupUniversitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
dc.description.peerreviewedPeer Reviewed
dc.rights.accessOpen Access
dc.description.versionPostprint (published version)
dc.relation.projectidinfo:eu-repo/grantAgreement/AGAUR/VPRI/2014 SGR 1051
upcommons.citation.authorVillalba, Á.; Berral, J.; Carrera, D.
upcommons.citation.publicationNameIEEE transactions on parallel and distributed systems

Files in this item


This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder