Mostra el registre d'ítem simple
Using annotations on Mechanical Turk to perform supervised polarity classification of Spanish customer comments
dc.contributor.author | Ruiz Costa-Jussà, Marta |
dc.contributor.author | Grivolla, Jens |
dc.contributor.author | Mellebeek, Bart |
dc.contributor.author | Benavent, Francesc |
dc.contributor.author | Codina, Joan |
dc.contributor.author | Codina, Joan |
dc.contributor.author | Banchs Martínez, Rafael Enrique |
dc.contributor.other | Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions |
dc.date.accessioned | 2016-02-04T15:29:19Z |
dc.date.issued | 2014-12-01 |
dc.identifier.citation | Costa-jussà, M. R., Grivolla, J., Mellebeek, B., Benavent, F., Codina, J., Codina, J., Banchs, R. Using annotations on Mechanical Turk to perform supervised polarity classification of Spanish customer comments. "Information sciences", 01 Desembre 2014, vol. 275, p. 400-412. |
dc.identifier.issn | 0020-0255 |
dc.identifier.uri | http://hdl.handle.net/2117/82570 |
dc.description.abstract | One of the major bottlenecks in the development of data-driven AI Systems is the cost of reliable human annotations. The recent advent of several crowdsourcing platforms such as Amazon’s Mechanical Turk, allowing requesters the access to affordable and rapid results of a global workforce, greatly facilitates the creation of massive training data. Most of the available studies on the effectiveness of crowdsourcing report on English data. We use Mechanical Turk annotations to train an Opinion Mining System to classify Spanish consumer comments. We design three different Human Intelligence Task (HIT) strategies and report high inter-annotator agreement between non-experts and expert annotators. We evaluate the advantages/drawbacks of each HIT design and show that, in our case, the use of non-expert annotations is a viable and cost-effective alternative to expert annotations. |
dc.format.extent | 13 p. |
dc.language.iso | eng |
dc.subject | Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial |
dc.subject.lcsh | Crowdsourcing |
dc.subject.other | Annotation on Mechanical Turk |
dc.subject.other | HIT design |
dc.subject.other | Supervised polarity classification |
dc.title | Using annotations on Mechanical Turk to perform supervised polarity classification of Spanish customer comments |
dc.type | Article |
dc.subject.lemac | Intel·ligència artificial |
dc.identifier.doi | 10.1016/j.ins.2014.01.043 |
dc.description.peerreviewed | Peer Reviewed |
dc.relation.publisherversion | http://www.sciencedirect.com/science/article/pii/S0020025514000796 |
dc.rights.access | Restricted access - publisher's policy |
local.identifier.drac | 17370681 |
dc.description.version | Postprint (published version) |
dc.date.lift | 10000-01-01 |
local.citation.author | Costa-jussà, M. R.; Grivolla, J.; Mellebeek, B.; Benavent, F.; Codina, J.; Codina, J.; Banchs, R. |
local.citation.publicationName | Information sciences |
local.citation.volume | 275 |
local.citation.startingPage | 400 |
local.citation.endingPage | 412 |
Fitxers d'aquest items
Aquest ítem apareix a les col·leccions següents
-
Articles de revista [2.526]