This paper tackles the supervised evaluation of image segmentation algorithms. First, it surveys and structures the measures used to compare the segmentation results with a ground truth database, and proposes a new measure: the precision-recall for objects and parts. To compare the goodness of these measures, it defines three quantitative meta-measures involving six state of the art segmentation methods. The meta-measures consist in assuming some plausible hypotheses about the results and assessing how well each measure reflects these hypotheses. As a conclusion, this paper proposes the precision-recall curves for boundaries and for objects-and-parts as the tool of choice for the supervised evaluation of image segmentation. We make the datasets and code of all the measures publicly available.
CitacióPont, J.; Marques, F. Measures and meta-measures for the supervised evaluation of image segmentation. A: IEEE Conference on Computer Vision and Pattern Recognition. "CVRP 2013: 2013 IEEE Conference on Computer Vision and Pattern Recognition: proceedings: 23-28 June 2013: Portland, Oregon, USA". Portland, Oregon: IEEE Computer Society Publications, 2013, p. 2131-2138.