Intermediate results materialization selection and format for data-intensive flows

Carregant...
Miniatura

Fitxers

Main Article (.pdf, 5.01 MB) (Accés restringit) Sol·licita una còpia a l'autor
El pots comprar en digital a:
El pots comprar en paper a:

Projectes de recerca

Unitats organitzatives

Número de la revista

Títol de la revista

ISSN de la revista

Títol del volum

Col·laborador

Tribunal avaluador

Realitzat a/amb

Tipus de document

Article

Data publicació

Editor

Condicions d'accés

Accés restringit per política de l'editorial

item.page.rightslicense

Tots els drets reservats. Aquesta obra està protegida pels drets de propietat intel·lectual i industrial corresponents. Sense perjudici de les exempcions legals existents, queda prohibida la seva reproducció, distribució, comunicació pública o transformació sense l'autorització de la persona titular dels drets

Assignatures relacionades

Assignatures relacionades

Publicacions relacionades

Datasets relacionats

Datasets relacionats

Projecte CCD

Abstract

Data-intensive flows deploy a variety of complex data transformations to build information pipelines from data sources to different end users. As data are processed, these workflows generate large intermediate results, typically pipelined from one operator to the following ones. Materializing intermediate results, shared among multiple flows, brings benefits not only in terms of performance but also in resource usage and consistency. Similar ideas have been proposed in the context of data warehouses, which are studied under the materialized view selection problem. With the rise of Big Data systems, new challenges emerge due to new quality metrics captured by service level agreements which must be taken into account. Moreover, the way such results are stored must be reconsidered, as different data layouts can be used to reduce the I/O cost. In this paper, we propose a novel approach for automatic selection of multi-objective materialization of intermediate results in data-intensive flows, which can tackle multiple and conflicting quality objectives. In addition, our approach chooses the optimal storage data format for selected materialized intermediate results based on subsequent access patterns. The experimental results show that our approach provides 40% better average speedup with respect to the current state-of-the-art, as well as an improvement on disk access time of 18% as compared to fixed format solutions.

Descripció

Persones/entitats

Document relacionat

item.page.versionof

Citació

Munir, R., Nadal, S., Romero, O., Abello, A., Jovanovic, P., Thiele, M., Lehner, W. Intermediate results materialization selection and format for data-intensive flows. "Fundamenta informaticae", 1 Maig 2018, vol. 163, núm. 3, p. 111-138.

Ajut

Forma part

Dipòsit legal

ISBN

ISSN

0169-2968

Altres identificadors

Referències