Biological color vision inspires artificial color processing

Simple cameras that use only a few bands do not offer the wonderful spectral discrimination capability of hyperspectral cameras that record a full spectrum at each pixel. Unfortunately, hyperspectral cameras are inevitably more complicated, more expensive, and less sensitive (because they share the available light per pixel among many detections) than simpler cameras. In nature, on the other hand, many creatures have excellent spectral discrimination using only a few channels. We can learn from the color vision of biological organisms how to produce artificial-color systems that have the discrimination capabilities of hyperspectral cameras as well as the simplicity and sensitivity of the throw-away cameras you can buy in a store.

In biological color, the scene is detected using broad, overlapping spectral-sensitivity curves. One curve won't do, because it produces no spectral contrast. That is why we see no color at night when only one kind of sensor cell (our rods) is working. Humans use from two to four types of cone cells to give color vision. We call those who use only two ‘color blind’ and those who use three ‘normal. ’ Perhaps 10% of people (all women) use four kinds of cone cells. Mantis shrimp use about a dozen kinds.

Our brains compute a Bayesian model of the object that must be out there in the world to have caused the detected pattern on our retinas. It uses both the pixels values with the various cone cells and the spatial content of the scene—plus expectations, fears, and so forth—to compute generally useful spectral discriminants and then it attributes them (now called colors) to the Bayesian model. We perceive that model. It is the percept, not the object, that has color. In the artificial-color method,^1,2 we use two or three spectrally-overlapping sensitivity curves to gather information, process the resulting data electronically, and attribute the discriminant to the object. Ideally, the sensitivity curves should be problem-specific, but I will illustrate here with the familiar RGB (red-green-blue) channels of a color camera.

Consider Figure 1, taken with an ordinary digital color camera (whose red, green, and blue filters overlap considerably). The frog is doing a very good job blending in with its background. We took 20 pixels at random from four regions: frog, dark green leaves, light green leaves, and dark shadows. Pattern-recognition experts will appreciate that 20 is a very small number, but it allows us to show how powerful artificial color is. We then trained our algorithm to recognize those pixels judged spectrally to belong to the class ‘frog but not dark leaves and not light leaves and not dark shadows.’ This is a very specialized color that will not be suitable for other problems. We then retained the image pixels with that artificial color and set the other pixels to white.

Figure 2 shows the frog extracted from its background using artificial color alone. Clearly we have recognized what amounts to an arbitrary set strictly on the basis of its red-green-blue components.

Of course, we do not have to work only in the visible. ³ We can design curves that allow useful discrimination in the infrared, for instance. In other work, we have improved the images with rank-order filters and with mathematical morphology,⁴ applied logic to the binary filters,⁵ applied fuzzy-logic techniques to the classification mechanism, and applied our artificial-color method successfully to test problems in biometrics,⁶ passport control, counterfeit currency detection, and oil discrimination.