Show simple item record

dc.contributor.authorCabrera-Bean, Margarita
dc.contributor.authorDiaz Vilor, Carles
dc.contributor.authorVidal Manzano, José
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions
dc.identifier.citationMargarita Cabrera-Bean, Diaz, C., Vidal, J. Impact of noisy annotators’ reliability in a crowdsourcing system performance. A: European Signal Processing Conference. "2016 24th European Signal Processing Conference (EUSIPCO) took place 28 August-2 September 2016 in Budapest, Hungary". Budapest: Institute of Electrical and Electronics Engineers (IEEE), 2016, p. 1-5.
dc.description.abstractCrowdsourcing is a powerful tool to harness citizen assessments in some complex decision tasks. When multiple annotators provide their individual labels a more reliable collective decision is obtained if the individual reliability parameters are incorporated in the decision making procedure. The well-known Maximum A Posteriori (MAP) rule weights the individual labels in proportion to the annotators’ reliability. In this work we analyze how the crowdsourcing system performance is degraded with the use of noisy annotators’ reliability parameters and we derive an alternative MAP based rule to be applied when these parameters are neither known nor even estimated by the decision system. We also derive analytical expected error rates and their upper bounds obtained by each rule as a useful tool to estimate the number of necessary annotators in the collective decision system depending on the level of noise present in the estimated reliability parameters.
dc.format.extent5 p.
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)
dc.subjectÀrees temàtiques de la UPC::Enginyeria de la telecomunicació::Processament del senyal
dc.subject.lcshSignal processing
dc.subject.otherExpected error rate bound
dc.titleImpact of noisy annotators’ reliability in a crowdsourcing system performance
dc.typeConference report
dc.subject.lemacTractament del senyal
dc.subject.lemacDecisió, Presa de
dc.contributor.groupUniversitat Politècnica de Catalunya. SPCOM - Grup de Recerca de Processament del Senyal i Comunicacions
dc.description.peerreviewedPeer Reviewed
dc.rights.accessRestricted access - publisher's policy
dc.description.versionPostprint (published version)
upcommons.citation.authorCabrera-Bean, Margarita; Diaz, C.; Vidal, J.
upcommons.citation.contributorEuropean Signal Processing Conference
upcommons.citation.publicationName2016 24th European Signal Processing Conference (EUSIPCO) took place 28 August-2 September 2016 in Budapest, Hungary

Files in this item


This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder