CSC 249/449 Computer Vision: Counting Colored Objects Assignment


The goal of this assignment is to count the number of objects of different types in an image, where the objects of different types have different characteristic colors. In this case, the objects are apples, oranges, and bananas, which (supposedly) have different colors (red, orange, and yellow) There are three test images on which you are to run your program in ~nelson/pics/fruit: mixed_fruit1.tiff , mixed_fruit2.tiff , and mixed_fruit3.tiff . There is also a very tough image fruit_tray.tiff , on which you can test the limits of your algorithm. In this picture, the goal is a little ambiguous, as there are some pieces of fruit of which only a small fragment is visible. By my count there are 10 - 11 oranges half or more visible plus 3 - 5 fragments, 8 apples plus 2 fragments, and 5 bananas plus one distinct fragment. Don't use the gifs here for testing - they are dithered pictures with 8 bits of color information, which will make your job even harder. There are also pictures of apples, oranges and bananas on a black background taken in daylight and fluorescent light, which might assist you in designing color models for the different categories.

The ipp utility library has routines input_image() and output_image() for loading and writing color iff and tiff images into an rgb core format, and the ipp graphics library routine display_image() will display an in-core (rgb) color image in x-windows. Note that the routines input_color_image(), output_color_image(), and display_color_image() refern to an obsolete color_image structure that has been superseded by augmenting the standard image structure with multi-band capability. See the utility Readme file for more information on the image format. Other than that, there is little software in the ipp libraries specifically for color images, so you will have to grow your own, or find some other routines.

The program xshow in my bin/(Solaris | PCLinux) will display a color iff or tiff image on an xterm and xv will also allow you to display color tiff images. the program ktv_get_color_image allows you to take a color image from the ktv digitizer (should you want to make additional image on your own), and the program bt_get_image with the -color option will get a color image from the BT848 digitizers on the Linux boxes. The bt_showlive program can also be used to take pictures by hitting the right mouse button inside the live image. This has the advantage that you can see what you are taking a picture of. Resolution and color mode are whatever bt_showlive is given.

There is a simple segmentation routine (con_comps) in the feature library that will group pixels that all have the same value, and might be of use to you if you manage to produce color labels on a pixel by pixel basis. There is also a gray-level region segmentation routine (grow_regions) that attemps a segmentation of a gray-level image. It won't do you much good directly, but you might find it useful as a model for a color segmentation scheme if you decide to take that route. Source for these routines is in ~nelson/programs/src/ipp/feature/lib/region.

You are permitted to use other software for low-level processing of color images (if you can find it), and for performing linear algebra and statistical operations (if you want them). You are NOT allowed to use a previously written color segmentation program, or higher level recognition software. This is what you are supposed to do yourself. (If you rewrite my gray level segmenter, you will have to understand it first, which is likely non-trivial, so that's ok).

Hand in the test images, separate images showing the fruit you found, and also a printed list giving the identity, location, and approximate size of all the objects you found. Also prepare a detailed writeup describing your algorithm (not the code), the difficulties you encoutered and how you resolved them, and evaluating the performance of your algorithm (how well did it do, what were the weaknesses, etc.).

Hints:

Back to vision course main page