Believe Your Eyes and Earworms

It is natural to assume that the visual information streaming into our eyes determines the visual perceptions we end up with, and that the auditory information entering our ears determines the events we hear. But the brain is more complicated than this. Visual and auditory information interact in the brain, and the brain utilizes both to guess what single scene to render a perception of. For example, the research of Ladan Shams, Yukiyasu Kamitani, and Shinsuke Shimojo at Caltech have shown that we perceive a single flash as a double flash if it is paired with a double beep. And Robert Sekuler and others from Brandeis University have shown that if a sound occurs at the time when the images of two balls pass through each other on a screen, the balls are instead perceived to have collided and reversed direction. These and other results of this kind demonstrate the interconnectedness of visual and auditory information in our brain. Visual ambiguity can be reduced by auditory information, and vice versa. And, generally, both are brought to bear in the brain’s attempt to guess about what’s out there.

Your brain, then, does not consist of independent visual and auditory systems, with separate troves of visual and auditory knowledge about the world. Instead, vision and audition talk to one another, and there are regions of cortex responsible for making vision and audition fit one another. These regions know about the sounds of looks and the looks of sounds. Because of this, when your brain hears something but cannot see it, your brain does not just sit there and refrain from guessing what it might have looked like. When your auditory system makes sense of something, it will have a tendency to activate visual areas, eliciting imagery of its best guess as to the appearance of the stuff making the sound. For example, when you hear the sound of your neighbor’s tree rustling, an image of its swaying, lanky branches may spring to mind. The mewing of your cat heard far away may evoke an image of it stuck high up in that tree. And the pumping of your neighbor’s kid’s BB gun can bring forth an image of the gun being pointed at Foofy way up there.

Your visual system, then, has strong opinions about the likely look of the things you hear. And, to get back to music, we can use the visual system’s strong opinions as an aid in gauging music’s meaning. In particular, we can ask your visual system what it thinks the appropriate visual is for music. If, for example, the visual system responds to music with images of beating hearts, then it would suggest, to my disbelief, that music mimics the sounds of heartbeats. If, instead, the visual system responds with images of pornography, then it would suggest that music sounds like sex. You get the idea.

But to get the visual system to act like an oracle, we need to get it to speak. How are we to know what the visual system thinks music looks like? One approach is to simply ask what visuals are routinely associated with music. For example, when people create imagery of musical notes, what does it look like? One cheap way to find out is simply to do a Google (or any search engine) image search on the term “musical notes.” You might think such a search would merely return images of simple notes on the page. However, that is not what one finds. To my surprise, actually, most of the images are like the one in Figure 16, with notes drawn in such a way that they appear to be moving through space. Notes in musical notation don’t look anything like this, and actual musical notes have no look at all (because they are sounds). And yet we humans seem prone to visually depict notes in lively motion.

Figure 16 . Musical notes tend to be visualized like this, a clue to their meaning.

Could these images of notes in motion be due to a more mundane association? Music is played by people, and people have to move to play their instruments. Could this be the source of the movement‑music association? I don’t think so, because the movement suggested in these images of notes doesn’t look anything like an instrument being played. In fact, it is common to show images of an instrument with the notes beginning their movement through space from the instrument: these notes are on their way somewhere, not tied to the musician’s key‑pressing or back‑and‑forth swaying.

Could it be that the musical notes are depicted as moving through space because sound waves move through space? The difficulty with this hypothesis is that all sound moves through space. All sound would, if this were so, be visually rendered as moving through space, but that’s not how we portray most sounds. For example, speech is not usually visually rendered as moving through space. Another difficulty is that the musical notes in these images are usually meandering, but sound waves don’t meander–sound waves go straight. A third problem with the notion that sound waves are the basis for the visual metaphor is that we never see sound waves in the first place.

Another possible counterhypothesis is that musical notes are visually depicted in motion because all auditory stimuli are caused by underlying events that involve movement of some kind. The first difficulty, as with sound waves, is that not all sound, by a long shot, is visually rendered as in motion. The second difficulty is that, while it is true that sounds are typically generated by movement of some kind, it need not be movement of an entire object through space. Moving parts within the object may make the noise, without the object going anywhere. In fact, the three examples I gave at the start of this section–leaves rustling, Foofy mewing, and the BB gun pumping–are noises without any bulk movement of the object (the tree, Foofy, or the BB gun, respectively). The musical notes in these images, on the other hand, really do seem to be moving their whole selves across space.

Music is like rustling leaves, Foofy, BB guns, and human speech, in that it is not made by bulk movements through space. And yet music appears uniquely likely to be visually depicted as notes moving through space. And not only moving, but meandering. When visually rendered, music looks alive and in motion (often along the ground)–just what one might expect if music’s secret is that it sounds like people moving.

A Google image search on “musical notes” is one way to try to discern what the visual system thinks music looks like. Another is simply to ask ourselves: what is the most common visual display shown during music? That is, if people were to make videos to go with music, what would the videos tend to look like? Luckily for us, people do make videos to go with music! They’re called music videos, of course. And what do they look like? The answer is so obvious that it hardly seems worth noting: music videos commonly show people moving about, usually in a manner that is time‑locked to the music, very often dancing. As obvious as it is that music videos typically show people moving, we must remember to ask ourselves why music isn’t typically visually associated with something very different. Why aren’t music videos mostly of rivers, avalanches, car races, windblown grass, lions hunting, fire, or bouncing balls? It is because, I am suggesting, our brain thinks that humans moving about is what music should look like . . . because it thinks that humans moving about is what music sounds like.

Musical notes are rendered as meandering through space. Music videos are built largely from people moving, and in a manner time‑locked to the music. That begins to suggest that the visual system is under the impression that music sounds like human movement. But if that’s really what the visual system thinks, then it should have more opinions than just “music sounds like movement.” It should have opinions about what kind of movement music sounds like, and therefore, more exactly what the movement should look like. Do our visual systems have opinions this precise? Are we picky about the visual movement that goes with music?

You bet we are! That’s choreography. It’s not OK to play a video of the Nutcracker ballet during Beatles music, nor is it OK to play a video of the Nutcracker to the music of Nutcracker , but with a small time lag between them. Video of human movement has to have all the right moves at the right time to be the right fit for music.

These strong opinions about what music looks like make perfect sense if music mimics human movement sounds. In real life, when people carry out complex behaviors, their visible movements are tightly choreographed with the sounds they make–because the sight and the sound arise from the same event. When you hear movement, you expect to see that same movement. Music sounds to your brain like human movement, and that’s why, when your brain hears music, it expects that any visual of it should match up with it.

We just used your brain’s visual system as an oracle to divine the meaning of music, and it answered, “People moving.” Let’s now use your brain in another oracle‑like fashion. If music has been culturally selected to fit the brain, then let’s look into which pieces of music are the best fit for the brain, with the idea that these pieces may be the best representatives of what music has been culturally selected to sound like. But how can we gauge which pieces of music are the best fits? One thought is that “symptoms” of a piece of music fitting the brain really well might be that the brain would process it especially easily, remember it easily, and internally hear it easily. Are there pieces of music like this?

Yes, there are! They’re called earworms –those songs with a tendency to get stuck in people’s heads. These pieces of music fit the brain so well that they can sometimes become nuisances. Earworms, then, may be great representatives of the fundamental structural features that have been selected for in music. What are the common qualities of pieces of music that become earworms?

When he was an RPI graduate student, Aaron Fath got interested in this question. He was dissatisfied with the standard line that songs become earworms because they are highly repetitive. Most songs are highly repetitive, he reasoned. Instead, he began to notice that a large fraction of earworms have a particular dance or move that goes along with the music. Examples of songs tightly connected to a particular movement include “I’m a Little Teacup,” “Macarena,” “YMCA,” “Chicken Dance,” “If You’re Happy and You Know It,” and “Head, Shoulders, Knees and Toes.” Let’s call these pieces movement‑explicit . He also noticed that many other earworms were songs that accompanied specific visual movements (like a commercial jingle on television) or were dance songs (even if no specific movements were associated with them).

Aaron used two existing catalogs of earworms: a top 17 list of earworms from James Kellaris of the University of Cincinnati (obtained by polling 559 students), and a list of “top annoying earworms” from an online poll at the website Keepers of Lists (one user posted 220 songs, and 80 other users voted on whether or not they were earwormy; Aaron took the 38 songs having more than 10 votes). Movement‑explicit pieces accounted for 23.5 percent and 18.4 percent of these lists. To gauge whether these are unusually large percentages of movement‑explicit pieces, he sampled the #8 song on the Billboard Hot 100 Chart every nine months from 1983 to the present, and among these 38 songs, none were of the movement‑explicit variety. As a second gauge, he sampled the #1 songs for each year from 1955 through 2006 (defined by Aaron–differently than Billboard does it–as the song released in a year that was #1 on the Billboard Hot 100 for the greatest number of weeks, and thus had the most staying power). Of these 52 songs, only one was of the movement‑explicit kind (namely, “Macarena”).

These data suggest that earworms are disproportionately movement‑explicit: about one‑fifth of the earworms had specific dance moves that went with them, whereas less than 2 percent of top pop songs are of this kind. Our speculation is that songs become earworms not because they are movement‑explicit so much as because they are consistent with the sounds of people moving –movement‑explicit songs just happen to be under especially strong selection pressure to be consistent with the sounds of people moving. Although only a fifth of the earworms were of the movement‑explicit kind, many of the others seemed to be in the “accompaniment” or “dance” category (although we have not yet tried to operationally measure these and compare them to control data sets). An alternative possibility is that when a song becomes tightly linked to movement, it is that very association that helps make it an earworm. This would suggest that music becomes more brain‑worthy when packaged together with a motor program, and this, too, would appear to point to the music‑is‑movement theory.

It looks like music may be the sounds of human movement. We asked the expert on how things look: your visual system. Like presenting a deeply encrypted code to an oracle, we asked for the visual system’s interpretation of that enigmatic thing called music, and it had a clear and resounding response: music sounds like people moving and doing things, and thus must be visually rendered as humanlike motion in sync with the musical sounds. We also queried your brain in another fashion: we asked it which songs it most revels in, which ones are so earwormalicious that the brain loves to internally sing them over and over again. And the brain answered: the more movement‑explicit songs are more likely to be the earwormy ones. The brain seems to be under the impression that music sounds like people moving.