Hearing colors, tasting shapes

When Matthew Blakeslee shapes hamburger patties with his hands, he experiences a vivid bitter taste in his mouth. Esmerelda Jones (a pseudonym) sees blue when she listens to the note C sharp played on the piano; other notes evoke different
hues—so much so that the piano keys are actually color-coded, making it easier for her to remember and play musical scales. And when Jeff Coleman looks atprinted black numbers, he sees them in color, each a different hue. Blakeslee,
Jones and Coleman are among a handful of otherwise normal people who have synesthesia. They experience the ordinary world in extraordinary ways and seem to inhabit a mysterious no-man’s-land between fantasy and reality. For them the sens- es—touch, taste, hearing, vision and smell—get mixed up in- stead of remainingseparate.
Modern scientists have known about synesthesia since 1880, when Francis Galton, a cousin of Charles Darwin, pub- lished a paper in Nature on the phenomenon. But most have brushed it aside as fakery, an artifact of drug use (LSD and mescaline can produce similar effects) or a mere curiosity. About four years ago, however, we and others began to un- cover brain processes that could account forsynesthesia. Along the way, we also found new clues to some of the most mysteri- ous aspects of the human mind, such as the emergence of ab- stract thought, metaphor and perhaps even language.
A common explanation of synesthesia is that the affected people are simply experiencing childhood memories and asso- ciations. Maybe a person had played with refrigerator magnets
as a child and the number5 was red and 6 was green. This the- ory does not answer why only some people retain such vivid sensory memories, however. You might think of cold when you look at a picture of an ice cube, but you probably do not feel cold, no matter how many encounters you may have had with ice and snow during your youth.
Another prevalent idea is that synesthetes are merely being metaphorical when theydescribe the note C flat as “red” or say that chicken tastes “pointy”—just as you and I might speak of a “loud” shirt or “sharp” cheddar cheese. Our ordinary lan- guage is replete with such sense-related metaphors, and perhaps synesthetes are just especially gifted in this regard.
We began trying to find out whether synesthesia is a gen- uine sensory experience in 1999. This deceptively simple ques-tion had plagued researchers in this field for decades. One nat- ural approach is to start by asking the subjects outright: “Is this just a memory, or do you actually see the color as if it were right in front of you?” When we tried asking this question, we did not get very far. Some subjects did respond, “Oh, I see it per-
www.sciam.com SCIENTIFIC AMERICAN 53
DAVID EMMITE
fectly clearly.” But amore frequent reac- tion was, “I kind of see it, kind of don’t” or “No, it is not like a memory. I see the number as being clearly red but I also know it isn’t; it’s black. So it must be a memory, I guess.”
To determine whether an effect is tru- ly perceptual, psychologists often use a simple test called pop-out or segregation. If you look at a set of tilted lines scattered amid a forest ofvertical lines, the tilted lines stand out. Indeed, you can instantly segregate them from the background and group them mentally to form, for exam- ple, a separate triangular shape. Similar- ly, if most of a background’s elements were green dots and you were told to look for red targets, the reds would pop out. On the other hand, a set of black 2’s scat- tered among 5’s of the same color almost blendin [see illustration on page 57]. It is hard to discern the 2’s without engaging in an item-by-item inspection of numbers, even though any individual number is just as clearly different from its neighbors as a tilted line is from a straight line. We thus may conclude that only certain primitive, or elementary, features, such as color and line orientation, can provide a basis for grouping. More...