Show simple item record

dc.contributor.authorCardoso Duarte, Amanda
dc.contributor.authorPalaskar, Shruti
dc.contributor.authorVentura Ripol, Lucas
dc.contributor.authorGhadiyaram, Deepti
dc.contributor.authorDeHaan, Kenneth
dc.contributor.authorMetze, Florian
dc.contributor.authorTorres Viñals, Jordi
dc.contributor.authorGiró Nieto, Xavier
dc.contributor.otherUniversitat Politècnica de Catalunya. Doctorat en Teoria del Senyal i Comunicacions
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions
dc.contributor.otherBarcelona Supercomputing Center
dc.date.accessioned2021-11-15T11:42:57Z
dc.date.available2021-11-15T11:42:57Z
dc.date.issued2021
dc.identifier.citationCardoso, A. [et al.]. How2Sign: A large-scale multimodal dataset for continuous American sign language. A: IEEE Conference on Computer Vision and Pattern Recognition. "2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition: Virtual, 19-25 June 2021: proceedings". Institute of Electrical and Electronics Engineers (IEEE), 2021, p. 2734-2743. ISBN 978-1-6654-4509-2. DOI 10.1109/CVPR46437.2021.00276.
dc.identifier.isbn978-1-6654-4509-2
dc.identifier.otherhttps://imatge.upc.edu/web/publications/how2sign-large-scale-multimodal-dataset-continuous-american-sign-language
dc.identifier.urihttp://hdl.handle.net/2117/356423
dc.description.abstractOne of the factors that have hindered progress in the areas of sign language recognition, translation, and production is the absence of large annotated datasets. Towards this end, we introduce How2Sign, a multimodal and multiview continuous American Sign Language (ASL) dataset, consisting of a parallel corpus of more than 80 hours of sign language videos and a set of corresponding modalities including speech, English transcripts, and depth. A three-hour subset was further recorded in the Panoptic studio enabling detailed 3D pose estimation. To evaluate the potential of How2Sign for real-world impact, we conduct a study with ASL signers and show that synthesized videos using our dataset can indeed be understood. The study further gives insights on challenges that computer vision should address in order to make progress in this field. Dataset website: http://how2sign.github.io/
dc.description.sponsorshipThis work received funding from Facebook through gifts to CMU and UPC; through projects TEC2016-75976-R, TIN2015- 65316-P, SEV-2015-0493 and PID2019-107255GB-C22 of the Spanish Government and 2017-SGR-1414 of Generalitat de Catalunya. This work used XSEDE’s “Bridges” system at the Pittsburgh Supercomputing Center (NSF award ACI- 1445606). Amanda Duarte has received support from la Caixa Foundation (ID 100010434) under the fellowship code LCF/BQ/IN18/11660029. Shruti Palaskar was supported by the Facebook Fellowship program.
dc.format.extent10 p.
dc.language.isoeng
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)
dc.subjectÀrees temàtiques de la UPC::Enginyeria de la telecomunicació::Processament del senyal::Processament de la imatge i del senyal vídeo
dc.subject.lcshAmerican Sign Language
dc.subject.lcshComputer vision
dc.subject.lcshMachine translating
dc.subject.otherSign language
dc.subject.otherDataset
dc.subject.otherMachine translation
dc.subject.otherThree-dimensional displays
dc.subject.otherAnnotations
dc.subject.otherPose estimation
dc.subject.otherGesture recognition
dc.subject.otherProduction
dc.subject.otherSpeech recognition
dc.titleHow2Sign: A large-scale multimodal dataset for continuous American sign language
dc.typeConference lecture
dc.subject.lemacLlengua de signes americana
dc.subject.lemacVisió per ordinador
dc.subject.lemacTraducció automàtica
dc.contributor.groupUniversitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
dc.contributor.groupUniversitat Politècnica de Catalunya. GPI - Grup de Processament d'Imatge i Vídeo
dc.identifier.doi10.1109/CVPR46437.2021.00276
dc.description.peerreviewedPeer Reviewed
dc.relation.publisherversionhttps://ieeexplore-ieee-org.recursos.biblioteca.upc.edu/document/9577749
dc.rights.accessOpen Access
local.identifier.drac32036125
dc.description.versionPostprint (author's final draft)
dc.relation.projectidinfo:eu-repo/grantAgreement/MINECO/1PE/TEC2016-75976-R
dc.relation.projectidinfo:eu-repo/grantAgreement/MINECO//TIN2015-65316-P/ES/COMPUTACION DE ALTAS PRESTACIONES VII/
dc.relation.projectidinfo:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PID2019-107255GB-C22/ES/UPC-COMPUTACION DE ALTAS PRESTACIONES VIII/
dc.relation.projectidinfo:eu-repo/grantAgreement/AGAUR/2017 SGR 1414
local.citation.authorCardoso, A.; Palaskar, S.; Ventura, L.; Ghadiyaram, D.; DeHaan, K.; Metze, F.; Torres, J.; Giró, X.
local.citation.contributorIEEE Conference on Computer Vision and Pattern Recognition
local.citation.publicationName2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition: Virtual, 19-25 June 2021: proceedings
local.citation.startingPage2734
local.citation.endingPage2743
dc.description.sdgObjectius de Desenvolupament Sostenible::10 - Reducció de les Desigualtats
dc.description.sdgObjectius de Desenvolupament Sostenible::4 - Educació de Qualitat::4.5 - Per a 2030, eliminar les disparitats de gènere en l’educació i garantir l’accés en condicions d’igualtat a les persones vulnerables, incloses les persones amb discapacitat, els pobles indígenes i els nens i nenes en situacions de vulnerabilitat, a tots els nivells de l’ensenyament i la formació professional
dc.description.sdgObjectius de Desenvolupament Sostenible::10 - Reducció de les Desigualtats::10.2 - Per a 2030, potenciar i promoure la inclusió social, econòmica i política de totes les persones, independentment de l’edat, sexe, discapacitat, raça, ètnia, origen, religió, situació econòmica o altra condició
dc.description.sdgObjectius de Desenvolupament Sostenible::4 - Educació de Qualitat


Files in this item

Thumbnail
Thumbnail
Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder