Ir al contenido (pulsa Retorno)

Universitat Politècnica de Catalunya

    • Català
    • Castellano
    • English
    • LoginRegisterLog in (no UPC users)
  • mailContact Us
  • world English 
    • Català
    • Castellano
    • English
  • userLogin   
      LoginRegisterLog in (no UPC users)

UPCommons. Global access to UPC knowledge

Banner header
69.147 UPC E-Prints
You are here:
View Item 
  •   DSpace Home
  • E-prints
  • Grups de recerca
  • inSSIDE - integrated Software, Service, Information and Data Engineering
  • Articles de revista
  • View Item
  •   DSpace Home
  • E-prints
  • Grups de recerca
  • inSSIDE - integrated Software, Service, Information and Data Engineering
  • Articles de revista
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Intermediate results materialization selection and format for data-intensive flows

Thumbnail
View/Open
Main Article (5,007Mb) (Restricted access)   Request copy 

Què és aquest botó?

Aquest botó permet demanar una còpia d'un document restringit a l'autor. Es mostra quan:

  • Disposem del correu electrònic de l'autor
  • El document té una mida inferior a 20 Mb
  • Es tracta d'un document d'accés restringit per decisió de l'autor o d'un document d'accés restringit per política de l'editorial
 
10.3233/FI-2018-1734
 
  View UPCommons Usage Statistics
  LA Referencia / Recolecta stats
Includes usage data since 2022
Cita com:
hdl:2117/125267

Show full item record
Munir, Rana FaisalMés informació
Nadal Francesch, SergiMés informacióMés informacióMés informació
Romero Moral, ÓscarMés informacióMés informacióMés informació
Abelló Gamazo, AlbertoMés informacióMés informacióMés informació
Jovanovic, PetarMés informacióMés informacióMés informació
Thiele, Maik
Lehner, Wolfgang
Document typeArticle
Defense date2018-05-01
Rights accessRestricted access - publisher's policy
All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder
Abstract
Data-intensive flows deploy a variety of complex data transformations to build information pipelines from data sources to different end users. As data are processed, these workflows generate large intermediate results, typically pipelined from one operator to the following ones. Materializing intermediate results, shared among multiple flows, brings benefits not only in terms of performance but also in resource usage and consistency. Similar ideas have been proposed in the context of data warehouses, which are studied under the materialized view selection problem. With the rise of Big Data systems, new challenges emerge due to new quality metrics captured by service level agreements which must be taken into account. Moreover, the way such results are stored must be reconsidered, as different data layouts can be used to reduce the I/O cost. In this paper, we propose a novel approach for automatic selection of multi-objective materialization of intermediate results in data-intensive flows, which can tackle multiple and conflicting quality objectives. In addition, our approach chooses the optimal storage data format for selected materialized intermediate results based on subsequent access patterns. The experimental results show that our approach provides 40% better average speedup with respect to the current state-of-the-art, as well as an improvement on disk access time of 18% as compared to fixed format solutions.
CitationMunir, R., Nadal, S., Romero, O., Abello, A., Jovanovic, P., Thiele, M., Lehner, W. Intermediate results materialization selection and format for data-intensive flows. "Fundamenta informaticae", 1 Maig 2018, vol. 163, núm. 3, p. 111-138. 
URIhttp://hdl.handle.net/2117/125267
DOI10.3233/FI-2018-1734
ISSN0169-2968
Publisher versionhttps://content.iospress.com/articles/fundamenta-informaticae/fi1734
Collections
  • inSSIDE - integrated Software, Service, Information and Data Engineering - Articles de revista [113]
  • Departament d'Enginyeria de Serveis i Sistemes d'Informació - Articles de revista [246]
  • GESSI - Grup d'Enginyeria del Software i dels Serveis - Articles de revista [56]
  • IMP - Information Modeling and Processing - Articles de revista [140]
  View UPCommons Usage Statistics

Show full item record

FilesDescriptionSizeFormatView
fi_journal.pdfBlockedMain Article5,007MbPDFRestricted access

Browse

This CollectionBy Issue DateAuthorsOther contributionsTitlesSubjectsThis repositoryCommunities & CollectionsBy Issue DateAuthorsOther contributionsTitlesSubjects

© UPC Obrir en finestra nova . Servei de Biblioteques, Publicacions i Arxius

info.biblioteques@upc.edu

  • About This Repository
  • Metadata under:Metadata under CC0
  • Contact Us
  • Send Feedback
  • Privacy Settings
  • Inici de la pàgina