Learning the semantics of object-action relations by observation
Cita com:
hdl:2117/14016
Tipus de documentArticle
Data publicació2011-10-28
Condicions d'accésAccés obert
Llevat que s'hi indiqui el contrari, els
continguts d'aquesta obra estan subjectes a la llicència de Creative Commons
:
Reconeixement-NoComercial-SenseObraDerivada 3.0 Espanya
ProjecteGARNICS - Gardening with a Cognitive System (EC-FP7-247947)
GARNICS - Gardening with a Cognitive System (EC-FP7-247947)
GARNICS - Gardening with a Cognitive System (EC-FP7-247947)
Abstract
Recognizing manipulations performed by a human and the transfer and execution of this by a robot is a difficult problem. We address this in the current study by introducing a novel representation of the relations between objects at decisive time points during a manipulation. Thereby, we encode the essential changes in a visual scenery in a condensed way such that a robot can recognize and learn a manipulation without prior object knowledge. To achieve this we continuously track image segments in the video and construct a dynamic graph sequence. Topological transitions of those graphs occur whenever a spatial relation between some segments has changed in a discontinuous way and these moments are stored in a transition matrix called the semantic event chain (SEC). We demonstrate that these time points are highly descriptive for distinguishing between different manipulations. Employing simple sub-string search algorithms, SECs can be compared and type-similar manipulations can be recognized with high confidence. As the approach is generic, statistical learning can be used to find the archetypal SEC of a given manipulation class. The performance of the algorithm is demonstrated on a set of real videos showing hands manipulating various objects and performing different actions. In experiments with a robotic arm, we show that the SEC can be learned by observing human manipulations, transferred to a new scenario, and then reproduced by the machine.
CitacióAksoy, E.E. [et al.]. Learning the semantics of object-action relations by observation. "International journal of robotics research", 28 Octubre 2011, vol. 30, núm. 10, p. 1229-1249.
ISSN0278-3649
Versió de l'editorhttp://dx.doi.org/10.1177/0278364911410459
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
1244-Learning-t ... lations-by-observation.pdf | 8,341Mb | Visualitza/Obre |