Mostrar el registro sencillo del ítem

dc.contributor.authorMunir, Rana Faisal
dc.contributor.authorNadal Francesch, Sergi
dc.contributor.authorRomero Moral, Óscar
dc.contributor.authorAbelló Gamazo, Alberto
dc.contributor.authorJovanovic, Petar
dc.contributor.authorThiele, Maik
dc.contributor.authorLehner, Wolfgang
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Enginyeria de Serveis i Sistemes d'Informació
dc.date.accessioned2018-11-30T08:29:46Z
dc.date.issued2018-05-01
dc.identifier.citationMunir, R., Nadal, S., Romero, O., Abello, A., Jovanovic, P., Thiele, M., Lehner, W. Intermediate results materialization selection and format for data-intensive flows. "Fundamenta informaticae", 1 Maig 2018, vol. 163, núm. 3, p. 111-138.
dc.identifier.issn0169-2968
dc.identifier.urihttp://hdl.handle.net/2117/125267
dc.description.abstractData-intensive flows deploy a variety of complex data transformations to build information pipelines from data sources to different end users. As data are processed, these workflows generate large intermediate results, typically pipelined from one operator to the following ones. Materializing intermediate results, shared among multiple flows, brings benefits not only in terms of performance but also in resource usage and consistency. Similar ideas have been proposed in the context of data warehouses, which are studied under the materialized view selection problem. With the rise of Big Data systems, new challenges emerge due to new quality metrics captured by service level agreements which must be taken into account. Moreover, the way such results are stored must be reconsidered, as different data layouts can be used to reduce the I/O cost. In this paper, we propose a novel approach for automatic selection of multi-objective materialization of intermediate results in data-intensive flows, which can tackle multiple and conflicting quality objectives. In addition, our approach chooses the optimal storage data format for selected materialized intermediate results based on subsequent access patterns. The experimental results show that our approach provides 40% better average speedup with respect to the current state-of-the-art, as well as an improvement on disk access time of 18% as compared to fixed format solutions.
dc.format.extent28 p.
dc.language.isoeng
dc.subjectÀrees temàtiques de la UPC::Informàtica::Sistemes d'informació
dc.subject.lcshBig data
dc.subject.otherBig Data
dc.subject.otherData-Intensive Flows
dc.subject.otherIntermediate Results
dc.subject.otherData Format
dc.subject.otherHDFS
dc.titleIntermediate results materialization selection and format for data-intensive flows
dc.typeArticle
dc.subject.lemacDades massives
dc.contributor.groupUniversitat Politècnica de Catalunya. inSSIDE - integrated Software, Service, Information and Data Engineering
dc.contributor.groupUniversitat Politècnica de Catalunya. IMP - Information Modeling and Processing
dc.identifier.doi10.3233/FI-2018-1734
dc.description.peerreviewedPeer Reviewed
dc.relation.publisherversionhttps://content.iospress.com/articles/fundamenta-informaticae/fi1734
dc.rights.accessRestricted access - publisher's policy
local.identifier.drac23535083
dc.description.versionPostprint (published version)
dc.date.lift10000-01-01
local.citation.authorMunir, R.; Nadal, S.; Romero, O.; Abello, A.; Jovanovic, P.; Thiele, M.; Lehner, W.
local.citation.publicationNameFundamenta informaticae
local.citation.volume163
local.citation.number3
local.citation.startingPage111
local.citation.endingPage138


Ficheros en el ítem

Imagen en miniatura

Este ítem aparece en la(s) siguiente(s) colección(ones)

Mostrar el registro sencillo del ítem

Todos los derechos reservados.Esta obra está protegida por los derechos de propiedad intelectual e industrial. Sin perjuicio de las exenciones legales existentes, queda prohibida su reproducción, distribución, comunicación pública o transformación sin la autorización del titular de los derechos