Data pre-processing pipeline generation for AutoETL
Visualitza/Obre
Cita com:
hdl:2117/362396
Tipus de documentArticle
Data publicació2022-09
EditorElsevier
Condicions d'accésAccés obert
Llevat que s'hi indiqui el contrari, els
continguts d'aquesta obra estan subjectes a la llicència de Creative Commons
:
Reconeixement-NoComercial-SenseObraDerivada 4.0 Internacional
Abstract
Data pre-processing plays a key role in a data analytics process (e.g., applying a classification algorithm on a predictive task). It encompasses a broad range of activities that span from correcting errors to selecting the most relevant features for the analysis phase. There is no clear evidence, or rules defined, on how pre-processing transformations impact the final results of the analysis. The problem is exacerbated when transformations are combined into pre-processing pipeline prototypes. Data scientists cannot easily foresee the impact of pipeline prototypes and hence require a method to discriminate between them and find the most relevant ones (e.g., with highest positive impact) for their study at hand. Once found, these prototypes can be instantiated and optimized e.g., using Bayesian Optimization. In this work, we study the impact of transformations when chained together into prototypes, and the impact of transformations when instantiated via various operators. We develop and scrutinize a generic method that allows to generate pre-processing pipelines, as a step towards AutoETL. We make use of rules that enable the construction of prototypes (i.e., define the order of transformations), and rules that guide the instantiation of the transformations inside the prototypes (i.e., define the operator for each transformation). The optimization of our effective pipeline prototypes provide results that compared to an exhaustive search, get 90% of the predictive accuracy in the median, but with a time cost that is 24 times smaller.
CitacióGiovanelli, J.; Bilalli, B.; Abelló, A. Data pre-processing pipeline generation for AutoETL. "Information systems", Setembre 2022, vol. 108, article 101957.
ISSN0306-4379
Versió de l'editorhttps://www.sciencedirect.com/science/article/pii/S0306437921001514
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
IS_2021.pdf | 3,611Mb | Visualitza/Obre |