Channel-wise early stopping without a validation set via NNK polytope interpolation
Visualitza/Obre
Estadístiques de LA Referencia / Recolecta
Inclou dades d'ús des de 2022
Cita com:
hdl:2117/366180
Tipus de documentText en actes de congrés
Data publicació2021
Condicions d'accésAccés obert
Tots els drets reservats. Aquesta obra està protegida pels drets de propietat intel·lectual i
industrial corresponents. Sense perjudici de les exempcions legals existents, queda prohibida la seva
reproducció, distribució, comunicació pública o transformació sense l'autorització del titular dels drets
Abstract
State-of-the-art neural network architectures continue to scale in size and deliver impressive generalization results, although this comes at the expense of limited interpretability. In particular, a key challenge is to determine when to stop training the model, as this has a significant impact on generalization. Convolutional neural networks (ConvNets) comprise high-dimensional feature spaces formed by the aggregation of multiple channels, where analyzing intermediate data representations and the model's evolution can be challenging owing to the curse of dimensionality. We present channel-wise DeepNNK (CW-DeepNNK), a novel channel-wise generalization estimate based on non-negative kernel regression (NNK) graphs with which we perform local polytope interpolation on low-dimensional channels. This method leads to instance-based interpretability of both the learned data representations and the relationship between channels. Motivated by our observations, we use CW-DeepNNK to propose a novel early stopping criterion that (i) does not require a validation set, (ii) is based on a task performance metric, and (iii) allows stopping to be reached at different points for each channel. Our experiments demonstrate that our proposed method has advantages as compared to the standard criterion based on validation set performance.
CitacióBonet, D. [et al.]. Channel-wise early stopping without a validation set via NNK polytope interpolation. A: Asia-Pacific Signal and Information Processing Association Annual Summit and Conference. "2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)". 2021, p. 351-358. ISBN 978-988-14768-9-0.
ISBN978-988-14768-9-0
Versió de l'editorhttps://ieeexplore.ieee.org/document/9689626
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
cBonet21.pdf | 1010,Kb | Visualitza/Obre |