The algorithm presented in this paper is comprised of three main stages: (1) classification of the image sequence and, in the case of a moving camera, parametric motion estimation, (2) change detection having as reference a fixed frame, an appropriately selected frame or a displaced frame, and (3) object localization using local colour features. The image sequence classification is based on statistical tests on the frame difference. The change detection module uses a two-label fast marching algorithm. Finally, the object localization uses a region growing algorithm based on the colour similarity. Video object segmentation results are shown using the COST 211 data set.