The objective of this work is to provide an efficient face recognition scheme useful for video indexing applications. In particular we are addressing the following problem: given a set of known images and given a video sequence to be indexed, find where the corresponding persons appear in the sequence. Conventional face detection schemes are not well suited for this application and alternate and more efficient schemes have to be developed. In this paper we have modified our original generic eigenface-based recognition scheme presented in [1] by introducing the concept of selfeigenfaces. The resulting scheme is very efficient to find specific face images and to cope with the different face conditions present in a video sequence. The main and final objective is to develop a tool to be used in the MPEG-7 standardization effort to help video indexing activities. Good results have been obtained using the video test sequences used in the MPEG-7 evaluation group.