Color-Based Object Recognition

Pattern Recognition 32 (1999) 453—464

Color-based object recognition
Theo Gevers*, Arnold W.M. Smeulders
ISIS, Faculty of WINS, University of Amsterdam, Kruislaan 403, 1098 SJ, Amsterdam, The Netherlands Received 22 December 1997; received for publication 4 February 1998

Abstract The purpose is to arrive at recognition of multicolored objects invariant to a substantial change inviewpoint, object geometry and illumination. Assuming dichromatic reflectance and white illumination, it is shown that normalized color rgb, saturation S and hue H, and the newly proposed color models c c c and l l l are all invariant to a change in     viewing direction, object geometry and illumination. Further, it is shown that hue H and l l l are also invariant to  highlights. Finally, achange in spectral power distribution of the illumination is considered to propose a new color constant color model m m m . To evaluate the recognition accuracy differentiated for the various color models,    experiments have been carried out on a database consisting of 500 images taken from 3-D multicolored man-made objects. The experimental results show that highest object recognition accuracy isachieved by l l l and hue H followed  by c c c , normalized color rgb and m m m under the constraint of white illumination. Also, it is demonstrated that       recognition accuracy degrades substantially for all color features other than m m m with a change in illumination    color. The recognition scheme and images are available within the PicToSeek and Pic2Seek systems on-line at:http: 1999 Pattern Recognition Society. Published by Elsevier Science Ltd. All //www.wins.uva.nl/research/isis/zomax/. rights reserved. Keywords: Object recognition; Multicolored objects; Color models; Dichromatic reflection; Reflectance properties; Photometric color invariants; Color constancy

1. Introduction Color provides powerful information for object recognition. A simple and effectiverecognition scheme is to represent and match images on the basis of color histograms as proposed by Swain and Ballard [1]. The work makes a significant contribution in introducing color for object recognition. However, it has the drawback that when the illumination circumstances are not equal, the object recognition accuracy degrades significantly. This method is extended by Funt and Finlayson [2],based on the retinex theory of Land [3], to make the method

*Corresponding author.

illumination independent by indexing on illuminationinvariant surface descriptors (color ratios) computed from neighboring points. However, it is assumed that neighboring points have the same surface normal. Therefore, the derived illumination-invariant surface descriptors are negatively affected by rapid changesin surface orientation of the object (i.e. the geometry of the object). Healey and Slater [4] and Finlayson et al. [5] use illumination-invariant moments of color distributions for object recognition. These methods are sensitive to object occlusion and cluttering as the moments are defined as an integral property on the object as one. In global methods, in general, occluded parts will disturbrecognition. Slater and Healey [6] circumvent this problem by computing the color features from small object regions instead of the entire object.

0031-3203/99/$ — See front matter 1999 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved. PII: S 0 0 3 1 - 3 2 0 3 ( 9 8 ) 0 0 0 3 6 - 3

454

T. Gevers, A.W.M. Smeulders / Pattern Recognition 32 (1999) 453–464From the above observations, the choice which color models to use does not only depend on their robustness against varying illumination across the scene (e.g. multiple light sources with different spectral power distributions), but also on their robustness against changes in surface orientation of the object (i.e. the geometry of the object), and on their robustness against object occlusion and...