Estimation of information in parallel Gaussian channels via model order selection
Document typeConference lecture
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Rights accessOpen Access
All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder
We study the problem of estimating the overall mutual information in M independent parallel discrete-time memory-less Gaussian channels from N independent data sample pairs per channel (inputs and outputs). We focus on the case where the number of active channels L is sparse in comparison with the total number of channels (L ≪ M), for which the direct application of the maximum likelihood principle is problematic due to overfitting, especially for moderate to small N. For this regime, we show that the bias of the mutual information estimate is reduced by resorting to the minimum description length (MDL) principle. As a result, simple pre-processing based on a per-channel threshold on the empirical squared correlation coefficient is required with a fixed threshold that monotonically decreases with N as 1 - N -1/N , for N ≥ 4. The resulting improvement is shown in terms of the estimated information bias.
CitationLópez, C.; De Cabrera, F.; Riba, J. Estimation of information in parallel Gaussian channels via model order selection. A: IEEE International Conference on Acoustics, Speech, and Signal Processing. "2020 IEEE International Conference on Acoustics, Speech,and Signal Processing: proceedings: May 4-8, 2020: Centre de Convencions Internacional de Barcelona (CCIB) Barcelona, Spain". Institute of Electrical and Electronics Engineers (IEEE), 2020, p. 5675-5679. ISBN 978-1-5090-6632-2. DOI 10.1109/ICASSP40776.2020.9053506.
|Estimation of Information in Parallel Gaussian.pdf||326,8Kb||View/Open|