Show simple item record

dc.contributor.authorButko, Taras
dc.contributor.authorCanton Ferrer, Cristian
dc.contributor.authorSegura Perales, Carlos
dc.contributor.authorGiró Nieto, Xavier
dc.contributor.authorNadeu Camprubí, Climent
dc.contributor.authorHernando Pericás, Francisco Javier
dc.contributor.authorCasas Pla, Josep Ramon
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions
dc.identifier.citationButko, T. [et al.]. Acoustic event detection based on feature-level fusion of audio and video modalities. "Eurasip journal on advances in signal processing", 15 Març 2011, vol. 2011, p. 1-11.
dc.descriptionResearch article
dc.description.abstractAcoustic event detection (AED) aims at determining the identity of sounds and their temporal position in audio signals. When applied to spontaneously generated acoustic events, AED based only on audio information shows a large amount of errors, which are mostly due to temporal overlaps. Actually, temporal overlaps accounted for more than 70% of errors in the realworld interactive seminar recordings used in CLEAR 2007 evaluations. In this paper, we improve the recognition rate of acoustic events using information from both audio and video modalities. First, the acoustic data are processed to obtain both a set of spectrotemporal features and the 3D localization coordinates of the sound source. Second, a number of features are extracted from video recordings by means of object detection, motion analysis, and multicamera person tracking to represent the visual counterpart of several acoustic events. A feature-level fusion strategy is used, and a parallel structure of binary HMM-based detectors is employed in our work. The experimental results show that information from both the microphone array and video cameras is useful to improve the detection rate of isolated as well as spontaneously generated acoustic events.
dc.format.extent11 p.
dc.subjectÀrees temàtiques de la UPC::Enginyeria de la telecomunicació::Processament del senyal::Processament de la parla i del senyal acústic
dc.subjectÀrees temàtiques de la UPC::Enginyeria de la telecomunicació::Processament del senyal::Processament de la imatge i del senyal vídeo
dc.subject.lcshAcoustic event detection
dc.titleAcoustic event detection based on feature-level fusion of audio and video modalities
dc.subject.lemacSenyal acústic -- Detecció
dc.contributor.groupUniversitat Politècnica de Catalunya. VEU - Grup de Tractament de la Parla
dc.contributor.groupUniversitat Politècnica de Catalunya. GPI - Grup de Processament d'Imatge i Vídeo
dc.description.peerreviewedPeer Reviewed
dc.rights.accessOpen Access
dc.description.versionPostprint (published version)
upcommons.citation.authorButko, T.; Canton-Ferrer, C.; Segura, C.; Giro, X.; Nadeu, C.; Hernando, J.; Casas, J.
upcommons.citation.publicationNameEurasip journal on advances in signal processing

Files in this item


This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder