An alternative view on data processing pipelines from the DOLAP 2019 perspective
Visualitza/Obre
Cita com:
hdl:2117/358649
Tipus de documentArticle
Data publicació2020-09
EditorElsevier
Condicions d'accésAccés obert
Llevat que s'hi indiqui el contrari, els
continguts d'aquesta obra estan subjectes a la llicència de Creative Commons
:
Reconeixement-NoComercial-SenseObraDerivada 4.0 Internacional
Abstract
Data science requires constructing data processing pipelines (DPPs), which span diverse phases such as data integration, cleaning, pre-processing, and analysis. However, current solutions lack a strong data engineering perspective. As consequence, DPPs are error-prone, inefficient w.r.t. human efforts, and inefficient w.r.t. execution time. We claim that DPP design, development, testing, deployment, and execution should benefit from a standardized DPP architecture and from well-known data engineering solutions. This claim is supported by our experience in real projects and trends in the field, and it opens new paths for research and technology. With this spirit, we outline five research opportunities that represent novel trends towards building DPPs. Finally, we highlight that the best DOLAP 2019 papers selected for the DOLAP 2019 Information Systems Special Issue fall in this category and highlight the relevance of advanced data engineering for data science.
CitacióRomero, O.; Wrembel, R.; Song, I. An alternative view on data processing pipelines from the DOLAP 2019 perspective. "Information systems", Setembre 2020, vol. 92, p. 1-4.
ISSN0306-4379
Versió de l'editorhttps://www.sciencedirect.com/science/article/pii/S0306437919305411
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
DOLAP_2019_IS.pdf | 238,8Kb | Visualitza/Obre |