Ir al contenido (pulsa Retorno)

Universitat Politècnica de Catalunya

    • Català
    • Castellano
    • English
    • LoginRegisterLog in (no UPC users)
  • mailContact Us
  • world English 
    • Català
    • Castellano
    • English
  • userLogin   
      LoginRegisterLog in (no UPC users)

UPCommons. Global access to UPC knowledge

Banner header
64.060 UPC academic works
You are here:
View Item 
  •   DSpace Home
  • Treballs acadèmics
  • Escola Tècnica Superior d'Enginyeria de Telecomunicació de Barcelona
  • Enginyeria de Telecomunicació (Pla 1992)
  • View Item
  •   DSpace Home
  • Treballs acadèmics
  • Escola Tècnica Superior d'Enginyeria de Telecomunicació de Barcelona
  • Enginyeria de Telecomunicació (Pla 1992)
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

FAPEC integration as an HDF5 filter

Thumbnail
View/Open
SD_CMP_H5.pdf (922,5Kb)
Share:
 
  View Usage Statistics
Cita com:
hdl:2117/90112

Show full item record
Dueñas Pedrosa, Sergi
Tutor / directorGarcía-Berro Montilla, EnriqueMés informació
Document typeMaster thesis (pre-Bologna period)
Date2016-07
Rights accessOpen Access
Attribution-NonCommercial-NoDerivs 3.0 Spain
Except where otherwise noted, content on this work is licensed under a Creative Commons license : Attribution-NonCommercial-NoDerivs 3.0 Spain
Abstract
In this work we propose a solution to some of the problems found in supercomputing environments by combining an extremely efficient, standard, open-source data manager suite with a high-performance data compressor. We do not intend to use such an efficient file format and, later, compress the resulting files or data sets without further ado, as we would be losing in the compression process the file format benefits. Our aim is to compress the little portions of data that conform the data sets stored inside the file (which are named chunks), thus without losing any of the functionalities offered by the mentioned data management suite. HDF5 is our choice for the data storage and management format, and FAPEC is the high-performance data compressor chosen. By integrating FAPEC as an HDF5 filter we will offer a solution that can solve in a smart, clean and efficient way the storage and management problems in supercomputing environments.
 
In this work we propose a solution to some of the problems found in supercomputing environments by combining an extremely efficient, standard, open-source data manager suite with a high-performance data compressor. We do not intend to use such an efficient file format and, later, compress the resulting files or data sets without further ado, as we would be losing in the compression process the file format benefits. Our aim is to compress the little portions of data that conform the data sets stored inside the file (which are named chunks), thus without losing any of the functionalities offered by the mentioned data management suite. HDF5 is our choice for the data storage and management format, and FAPEC is the high-performance data compressor chosen. By integrating FAPEC as an HDF5 filter we will offer a solution that can solve in a smart, clean and efficient way the storage and management problems in supercomputing environments.
 
In this work we propose a solution to some of the problems found in supercomputing environments by combining an extremely efficient, standard, open-source data manager suite with a high-performance data compressor. We do not intend to use such an efficient file format and, later, compress the resulting files or data sets without further ado, as we would be losing in the compression process the file format benefits. Our aim is to compress the little portions of data that conform the data sets stored inside the file (which are named chunks), thus without losing any of the functionalities offered by the mentioned data management suite. HDF5 is our choice for the data storage and management format, and FAPEC is the high-performance data compressor chosen. By integrating FAPEC as an HDF5 filter we will offer a solution that can solve in a smart, clean and efficient way the storage and management problems in supercomputing environments.
Description
The Data Compression Group of the Institute for Space Studies of Catalonia (IEEC) has developed FAPEC, the Fully Adaptive Prediction Error Coder. It is a highly optimized adaptive entropy coder, which can be applied as a data compression solution for satellite payloads owing to its very quick and autonomous operation, furthermore with a high resiliency in front of data outliers. FAPEC is also being prepared for on-ground applications as well, for instance within HDF5, a general purpose library and file format for storing scientific data. The project proposed here consists i
SubjectsCompressors, Data compression (Telecommunication), Compressors, Dades -- Compressió (Telecomunicació)
DegreeENGINYERIA DE TELECOMUNICACIÓ (Pla 1992)
URIhttp://hdl.handle.net/2117/90112
Collections
  • Escola Tècnica Superior d'Enginyeria de Telecomunicació de Barcelona - Enginyeria de Telecomunicació (Pla 1992) [1.590]
Share:
 
  View Usage Statistics

Show full item record

FilesDescriptionSizeFormatView
SD_CMP_H5.pdf922,5KbPDFView/Open

Browse

This CollectionBy Issue DateAuthorsOther contributionsTitlesSubjectsThis repositoryCommunities & CollectionsBy Issue DateAuthorsOther contributionsTitlesSubjects

© UPC Obrir en finestra nova . Servei de Biblioteques, Publicacions i Arxius

info.biblioteques@upc.edu

  • About This Repository
  • Contact Us
  • Send Feedback
  • Privacy Settings
  • Inici de la pàgina