Evaluating the benefits of key-value databases for scientific applications
Visualitza/Obre
10.1007/978-3-030-22734-0_30
Inclou dades d'ús des de 2022
Cita com:
hdl:2117/166407
Tipus de documentText en actes de congrés
Data publicació2019
EditorSpringer
Condicions d'accésAccés obert
Tots els drets reservats. Aquesta obra està protegida pels drets de propietat intel·lectual i
industrial corresponents. Sense perjudici de les exempcions legals existents, queda prohibida la seva
reproducció, distribució, comunicació pública o transformació sense l'autorització del titular dels drets
ProjecteCOMPUTACION DE ALTAS PRESTACIONES VII (MINECO-TIN2015-65316-P)
HBP SGA1 - Human Brain Project Specific Grant Agreement 1 (EC-H2020-720270)
HBP SGA2 - Human Brain Project Specific Grant Agreement 2 (EC-H2020-785907)
HBP SGA1 - Human Brain Project Specific Grant Agreement 1 (EC-H2020-720270)
HBP SGA2 - Human Brain Project Specific Grant Agreement 2 (EC-H2020-785907)
Abstract
The convergence of Big Data applications with High-Performance Computing requires new methodologies to store, manage and process large amounts of information. Traditional storage solutions are unable to scale and that results in complex coding strategies. For example, the brain atlas of the Human Brain Project has the challenge to process large amounts of high-resolution brain images. Given the computing needs, we study the effects of replacing a traditional storage system with a distributed Key-Value database on a cell segmentation application. The original code uses HDF5 files on GPFS through an intricate interface, imposing synchronizations. On the other hand, by using Apache Cassandra or ScyllaDB through Hecuba, the application code is greatly simplified. Thanks to the Key-Value data model, the number of synchronizations is reduced and the time dedicated to I/O scales when increasing the number of nodes.
CitacióSantamaria, P. [et al.]. Evaluating the benefits of key-value databases for scientific applications. A: International Conference on Computational Science. "Computational Science: ICCS 2019, 19th International Conference: Faro, Portugal, June 12–14, 2019: proceedings, part I". Berlín: Springer, 2019, p. 412-426.
ISBN3030227340
Versió de l'editorhttps://link.springer.com/chapter/10.1007/978-3-030-22734-0_30
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
Evaluating the benefits.pdf | 692,6Kb | Visualitza/Obre |