Whether it’s Wall Street bankers justifying million-dollar bonuses, or educational reformers stumping for higher teacher pay, the refrain is familiar: big money ensures high-quality work. Duncan Watts doesn’t buy it. In April, the Columbia University sociology professor and Yahoo Research scientist came to campus as part of Penn Engineering’s “Market and Social Systems” lecture series, bringing along some experimental data that cast doubt on the notion that salary bumps actually beget better performance.
Like much of Watts’ work, it came from a medium that social scientists hardly dreamed of using until fairly recently: the Internet. To test the hypothesis that financial bonuses elevate work quality, he and a colleague at Yahoo Research set up a website modeled on Amazon’s “Mechanical Turk” service, which pays users a small fee to perform the sort of tasks that are hard for computers, like labeling a stack of photos. Watts downloaded a collection of free images taken by traffic cameras at 2-second intervals, jumbled them, then paid visitors to put them back in order. This turns out to be “actually quite difficult to do when the traffic is moving slowly,” he says.
Without their knowledge, each user was assigned to one of three pay grades, and difficulty levels. Then, on top of their salary, participants were promised a “bonus” for more accurate work.
Predictably, the financial terms got their attention. “There’s a very clear, linear relationship” between the amount each user was paid and the amount of work they agreed to do, Watts said. Without a doubt, “people are paying attention to the money.”
But he found a surprising disconnect between pay and the actual quality of the work. “People who are getting paid more are staying with the task” longer, he said, but the bonus had no statistically measurable effect on accuracy. For those already doing a bad job, the bonus simply motivated them to turn in an even larger volume of mediocre work.
While this may not jell with Wall Street’s bonus theory, it’s just the sort of thing that piques Watts’ experimental curiosity. Another question he’s tackled, with a little help from the Web, is whether pop sensations like Madonna really earn their fame, or whether they just happen to be in the right place at the right time.
To find out what makes a pop hit, his research team set up a “Music Lab” website, where visitors could listen to a collection of unknown songs for free, then download the ones they liked. Some of the listeners did this in a social vacuum, picking songs on merit alone. But others could see how many times a song had already been downloaded—that is, how popular it had already become. The experiment ran eight times, which allowed Watts and his team to “re-run history,” as he terms it, and see whether the same songs would always rise to the top.
The data suggested that Madonna might be more lucky than good. Good songs did better than average, and truly terrible music never took off. But the researchers found a powerful tipping point where a small, initial burst of popularity can feed back into itself, propelling an otherwise ordinary song to the top of the chart. Each time he ran the experiment, a different top song emerged among listeners who could see what others had chosen.
Sociologists have studied patterns of group behavior for decades, but until recently they’ve been limited to low-tech forms of observation, questionnaires, and phone surveys that made truly large-scale research projects prohibitively expensive. In the late 1990s, when Watts was a grad student, “we were pretty pleased with ourselves if we could come up with a network of 2,000 individuals.”
Now the Web itself is his scientific instrument. Watts draws a comparison to the 16th century, when Galileo first saw the sky through a telescope. Suddenly, Watts says, “we were able to render visible things which had previously been invisible”—and celestial mechanics was born. What he is doing now is probing the “dark matter” of group dynamics and social interaction. With the explosive growth of social networking services like Facebook, which claims 200 million users who have formed 22 billion virtual friendships, the numbers are now cosmic in scale. And instead of relying on survey questions for data, social scientists have a way to observe complex and interactive behavior directly—at least in the virtual realm.
—Sean Whiteman LPS’11
