The Rise of Crowdsourcing

"Technological advances in everything from product design software to digital video cameras are breaking down the cost barriers that once separated amateurs from professionals. Hobbyists, part-timers, and dabblers suddenly have a market for their efforts, as smart companies in industries as disparate as pharmaceuticals and television discover ways to tap the latent talent of the crowd. . . . . "

Claudia Menashe needed pictures of sick people. A project director at the National Health Museum in Washington, DC, Menashe was putting together a series of interactive kiosks devoted to potential pandemics like the avian flu. An exhibition designer had created a plan for the kiosk itself, but now Menashe was looking for images to accompany the text. Rather than hire a photographer to take shots of people suffering from the flu, Menashe decided to use preexisting images --stock photography, as it's known in the publishing industry.

In October 2004, she ran across a stock photo collection by Mark Harmel, a freelance photographer living in Manhattan Beach, California. Harmel, whose wife is a doctor, specializes in images related to the health care industry. "Claudia wanted people sneezing, getting immunized, that sort of thing," recalls Harmel, a slight, soft-spoken 52-year-old.

The National Health Museum has grand plans to occupy a spot on the National Mall in Washington by 2012, but for now it's a fledgling institution with little money. "They were on a tight budget, so I charged them my nonprofit rate," says Harmel, who works out of a cozy but crowded office in the back of the house he shares with his wife and stepson. He offered the museum a generous discount: $100 to $150 per photograph. "That's about half of what a corporate client would pay," he says. Menashe was interested in about four shots, so for Harmel, this could be a sale worth $600.

After several weeks of back-and-forth, Menashe emailed Harmel to say that, regretfully, the deal was off. "I discovered a stock photo site called iStockphoto," she wrote, "which has images at very affordable prices." That was an understatement. The same day, Menashe licensed 56 pictures through iStockphoto --for about $1 each.

iStockphoto, which grew out of a free image-sharing exchange used by a group of graphic designers, had undercut Harmel by more than 99 percent. How? By creating a marketplace for the work of amateur photographers --homemakers, students, engineers, dancers. There are now about 22,000 contributors to the site, which charges between $1 and $5 per basic image. (Very large, high-resolution pictures can cost up to $40.) Unlike professionals, iStockers don't need to clear $130,000 a year from their photos just to break even; an extra $130 does just fine. "I negotiate my rate all the time," Harmel says. "But how can I compete with a dollar?"

He can't, of course. For Harmel, the harsh economics lesson was clear: The product Harmel offers is no longer scarce. Professional-grade cameras now cost less than $1,000. With a computer and a copy of Photoshop, even entry-level enthusiasts can create photographs rivaling those by professionals like Harmel. Add the Internet and powerful search technology, and sharing these images with the world becomes simple.

At first, the stock industry aligned itself against iStockphoto and other so-called microstock agencies like ShutterStock and Dreamstime. Then, in February, Getty Images, the largest agency by far with more than 30 percent of the global market, purchased iStockphoto for $50 million. "If someone's going to cannibalize your business, better it be one of your other businesses," says Getty CEO Jonathan Klein. iStockphoto's revenue is growing by about 14 percent a month and the service is on track to license about 10 million images in 2006 --several times what Getty's more expensive stock agencies will sell. iStockphoto's clients now include bulk photo purchasers like IBM and United Way, as well as the small design firms once forced to go to big stock houses. "I was using Corbis and Getty, and the image fees came out of my design fees, which kept my margin low," notes one UK designer in an email to the company. "iStockphoto's micro-payment system has allowed me to increase my profit margin." Welcome to the age of the crowd. Just as distributed computing projects like UC Berkeley's SETI@home have tapped the unused processing power of millions of individual computers, so distributed labor networks are using the Internet to exploit the spare processing power of millions of human brains. The open source software movement proved that a network of passionate, geeky volunteers could write code just as well as the highly paid developers at Microsoft or Sun Microsystems. Wikipedia showed that the model could be used to create a sprawling and surprisingly comprehensive online encyclopedia. And companies like eBay and MySpace have built profitable businesses that couldn't exist without the contributions of users.

All these companies grew up in the Internet age and were designed to take advantage of the networked world. But now the productive potential of millions of plugged-in enthusiasts is attracting the attention of old-line businesses, too. For the last decade or so, companies have been looking overseas, to India or China, for cheap labor. But now it doesn't matter where the laborers are --they might be down the block, they might be in Indonesia --as long as they are connected to the network.

Technological advances in everything from product design software to digital video cameras are breaking down the cost barriers that once separated amateurs from professionals. Hobbyists, part-timers, and dabblers suddenly have a market for their efforts, as smart companies in industries as disparate as pharmaceuticals and television discover ways to tap the latent talent of the crowd. The labor isn't always free, but it costs a lot less than paying traditional employees. It's not outsourcing; it's crowdsourcing.

It took a while for Harmel to recognize what was happening. "When the National Health Museum called, I'd never heard of iStockphoto," he says. "But now, I see it as the first hole in the dike." In 2000, Harmel made roughly $69,000 from a portfolio of 100 stock photographs, a tidy addition to what he earned from commissioned work. Last year his stock business generated less money --$59,000 --from more than 1,000 photos. That's quite a bit more work for less money.

Harmel isn't the only photographer feeling the pinch. Last summer, there was a flurry of complaints on the Stock Artists Alliance online forum. "People were noticing a significant decline in returns on their stock portfolios," Harmel says. "I can't point to iStockphoto and say it's the culprit, but it has definitely put downward pressure on prices." As a result, he has decided to shift the focus of his business to assignment work. "I just don't see much of a future for professional stock photography," he says.

2. The Packager

"Is that even a real horse? It looks like it doesn't have any legs," says Michael Hirschorn, executive vice president of original programming and production at VH1 and a creator of the cable channel's hit show Web Junk 20. The program features the 20 most popular videos making the rounds online in any given week. Hirschorn and the rest of the show's staff are gathered in the artificial twilight of a VH1 editing room, reviewing their final show of the season. The horse in question is named Patches, and it's sitting in the passenger seat of a convertible at a McDonald's drive-through window. The driver orders a cheeseburger for Patches. "Oh, he's definitely real," a producer replies. "We've got footage of him drinking beer." The crew breaks into laughter, and Hirschorn asks why they're not using that footage. "Standards didn't like it," a producer replies. Standards --aka Standards and Practices, the people who decide whether a show violates the bounds of taste and decency --had no such problem with Elvis the Robocat or the footage of a bicycle racer being attacked by spectators and thrown violently from a bridge. Web Junk 20 brings viewers all that and more, several times a week. In the new, democratic age of entertainment by the masses, for the masses, stupid pet tricks figure prominently.

The show was the first regular program to repackage the Internet's funniest home videos, but it won't be the last. In February, Bravo launched a series called Outrageous and Contagious: Viral Videos, and USA Network has a similar effort in the works. The E! series The Soup has a segment called "Cybersmack," and NBC has a pilot in development hosted by Carson Daly called Carson Daly's Cyberhood, which will attempt to bring beer-drinking farm animals to the much larger audiences of network TV. Al Gore's Current TV is placing the most faith in the model: More than 30 percent of its programming consists of material submitted by viewers.

Viral videos are a perfect fit for VH1, which knows how to repurpose content to make compelling TV on a budget. The channel reinvented itself in 1996 as a purveyor of tawdry nostalgia with Pop-Up Video and perfected the form six years later with I Love the 80s. "That show was a good model because it got great ratings, and we licensed the clips" --quick hits from such cultural touchstones as The A-Team and Fatal Attraction --"on the cheap," Hirschorn says. (Full disclosure: I once worked for Hirschorn at Inside.com.) But the C-list celebrity set soon caught on to VH1's searing brand of ridicule. "It started to get more difficult to license the clips," says Hirschorn, who has the manner of a laid-back English professor. "And we're spending more money now to get them, as our ratings have improved."

But Hirschorn knew of a source for even more affordable clips. He had been watching the growth of video on the Internet and figured there had to be a way to build a show around it. "I knew we offered something YouTube couldn't: television," he says. "Everyone wants to be on TV." At about the same time, VH1's parent company, Viacom, purchased iFilm --a popular repository of video clips --for $49 million. Just like that, Hirschorn had access to a massive supply of viral videos. And because iFilm already ranks videos by popularity, the service came with an infrastructure for separating the gold from the god-awful. The model's most winning quality, as Hirschorn readily admits, is that it's "incredibly cheap" --cheaper by far than anything else VH1 produces, which is to say, cheaper than almost anything else on television. A single 30-minute episode costs somewhere in the mid-five figures --about a tenth of what the channel pays to produce so noTORIous, a scripted comedy featuring Tori Spelling that premiered in April. And if the model works on a network show like Carson Daly's Cyberhood, the savings will be much greater: The average half hour of network TV comedy now costs nearly $1 million to produce.

Web Junk 20 premiered in January, and ratings quickly exceeded even Hirschorn's expectations. In its first season, the show is averaging a respectable half-million viewers in the desirable 18-to-49 age group, which Hirschorn says is up more than 40 percent from the same Friday-night time slot last year. The numbers helped persuade the network to bring Web Junk 20 back for another season.

Hirschorn thinks the crowd will be a crucial component of TV 2.0. "I can imagine a time when all of our shows will have a user-generated component," he says. The channel recently launched Air to the Throne, an online air guitar contest, in which viewers serve as both talent pool and jury. The winners will be featured during the VH1 Rock Honors show premiering May 31. Even VH1's anchor program, Best Week Ever, is including clips created by viewers.

But can the crowd produce enough content to support an array of shows over many years? It's something Brian Graden, president of entertainment for MTV Music Networks Group, is concerned about. "We decided not to do 52 weeks a year of Web Junk, because we don't want to burn the thing," he says. Rather than relying exclusively on the supply of viral clips, Hirschorn has experimented with soliciting viewers to create videos expressly for Web Junk 20. Early results have been mixed. Viewers sent in nearly 12,000 videos for the Show Us Your Junk contest. "The response rate was fantastic," says Hirschorn as he and other staffers sit in the editing room. But, he adds, "almost all of them were complete crap."

Choosing the winners, in other words, was not so difficult. "We had about 20 finalists." But Hirschorn remains confident that as user-generated TV matures, the users will become more proficient and the networks better at ferreting out the best of the best. The sheer force of consumer behavior is on his side. Late last year the Pew Internet & American Life Project released a study revealing that 57 percent of 12- to 17-year-olds online --12 million individuals --are creating content of some sort and posting it to the Web. "Even if the signal-to-noise ratio never improves --which I think it will, by the way --that's an awful lot of good material," Hirschorn says. "I'm confident that in the end, individual pieces will fail but the model will succeed."

3. The Tinkerer

The future of corporate R&D can be found above Kelly's Auto Body on Shanty Bay Road in Barrie, Ontario. This is where Ed Melcarek, 57, keeps his "weekend crash pad," a one-bedroom apartment littered with amplifiers, a guitar, electrical transducers, two desktop computers, a trumpet, half of a pontoon boat, and enough electric gizmos to stock a RadioShack. On most Saturdays, Melcarek comes in, pours himself a St. Remy, lights a Player cigarette, and attacks problems that have stumped some of the best corporate scientists at Fortune 100 companies.

Not everyone in the crowd wants to make silly videos. Some have the kind of scientific talent and expertise that corporate America is now finding a way to tap. In the process, forward-thinking companies are changing the face of R&D. Exit the white lab coats; enter Melcarek --one of over 90,000 "solvers" who make up the network of scientists on InnoCentive, the research world's version of iStockphoto.

Pharmaceutical maker Eli Lilly funded InnoCentive's launch in 2001 as a way to connect with brainpower outside the company --people who could help develop drugs and speed them to market. From the outset, InnoCentive threw open the doors to other firms eager to access the network's trove of ad hoc experts. Companies like Boeing, DuPont, and Procter & Gamble now post their most ornery scientific problems on InnoCentive's Web site; anyone on InnoCentive's network can take a shot at cracking them.

The companies --or seekers, in InnoCentive parlance --pay solvers anywhere from $10,000 to $100,000 per solution. (They also pay InnoCentive a fee to participate.) Jill Panetta, InnoCentive's chief scientific officer, says more than 30 percent of the problems posted on the site have been cracked, "which is 30 percent more than would have been solved using a traditional, in-house approach."

The solvers are not who you might expect. Many are hobbyists working from their proverbial garage, like the University of Dallas undergrad who came up with a chemical to use inart restoration, or the Cary, North Carolina, patent lawyer who devised a novel way to mix large batches of chemical compounds.

This shouldn't be surprising, notes Karim Lakhani, a lecturer in technology and innovation at MIT, who has studied InnoCentive. "The strength of a network like InnoCentive's is exactly the diversity of intellectual background," he says. Lakhani and his three coauthors surveyed 166 problems posted to InnoCentive from 26 different firms. "We actually found the odds of a solver's success increased in fields in which they had no formal expertise," Lakhani says. He has put his finger on a central tenet of network theory, what pioneering sociologist Mark Granovetter describes as "the strength of weak ties." The most efficient networks are those that link to the broadest range of information, knowledge, and experience.

Which helps explain how Melcarek solved a problem that stumped the in-house researchers at Colgate-Palmolive. The giant packaged goods company needed a way to inject fluoride powder into a toothpaste tube without it dispersing into the surrounding air. Melcarek knew he had a solution by the time he'd finished reading the challenge: Impart an electric charge to the powder while grounding the tube. The positively charged fluoride particles would be attracted to the tube without any significant dispersion.

"It was really a very simple solution," says Melcarek. Why hadn't Colgate thought of it? "They're probably test tube guys without any training in physics." Melcarek earned $25,000 for his efforts. Paying Colgate-Palmolive's R&D staff to produce the same solution could have cost several times that amount --if they even solved it at all. Melcarek says he was elated to win. "These are rocket-science challenges," he says. "It really reinforced my confidence in what I can do."

Melcarek, who favors thick sweaters and a floppy fishing hat, has charted an unconventional course through the sciences. He spent four years earning his master's degree at the world-class particle accelerator in Vancouver, British Columbia, but decided against pursuing a PhD. "I had an offer from the private sector," he says, then pauses. "I really needed the money." A succession of "unsatisfying" engineering jobs followed, none of which fully exploited Melcarek's scientific training or his need to tinker. "I'm not at my best in a 9-to-5 environment," he says. Working sporadically, he has designed products like heating vents and industrial spray-painting robots. Not every quick and curious intellect can land a plum research post at a university or privately funded lab. Some must make HVAC systems.

For Melcarek, InnoCentive has been a ticket out of this scientific backwater. For the past three years, he has logged onto the network's Web site a few times a week to look at new problems, called challenges. They are categorized as either chemistry or biology problems. Melcarek has formal training in neither discipline, but he quickly realized this didn't hinder him when it came to chemistry. "I saw that a lot of the chemistry challenges could be solved using electromechanical processes I was familiar with from particle physics," he says. "If I don't know what to do after 30 minutes of brainstorming, I give up." Besides the fluoride injection challenge, Melcarek also successfully came up with a method for purifying silicone-based solvents. That challenge paid $10,000. Other Melcarek solutions have been close runners-up, and he currently has two more up for consideration. "Not bad for a few weeks' work," he says with a chuckle.

It's also not a bad deal for the companies that can turn to the crowd to help curb the rising cost of corporate research. "Everyone I talk to is facing a similar issue in regards to R&D," says Larry Huston, Procter & Gamble's vice president of innovation and knowledge. "Every year research budgets increase at a faster rate than sales. The current R&D model is broken."

Huston has presided over a remarkable about-face at P&G, a company whose corporate culture was once so insular it became known as "the Kremlin on the Ohio." By 2000, the company's research costs were climbing, while sales remained flat. The stock price fell by more than half, and Huston led an effort to reinvent the way the company came up with new products. Rather than cut P&G's sizable in-house R&D department (which currently employs 9,000 people), he decided to change the way they worked.

Seeing that the company's most successful products were a result of collaboration between different divisions, Huston figured that even more cross-pollination would be a good thing. Meanwhile, P&G had set a goal of increasing the number of innovations acquired from outside its walls from 15 percent to 50 percent. Six years later, critical components of more than 35 percent of the company's initiatives were generated outside P&G. As a result, Huston says, R&D productivity is up 60 percent, and the stock has returned to five-year highs. "It has changed how we define the organ-ization," he says. "We have 9,000 people on our R&D staff and up to 1.5 million researchers working through our external networks. The line between the two is hard to draw."

P&G is one of InnoCentive's earliest and best customers, but the company works with other crowdsourcing networks as well. YourEncore, for example, allows companies to find and hire retired scientists for one-off assignments. NineSigma is an online marketplace for innovations, matching seeker companies with solvers in a marketplace similar to InnoCentive. "People mistake this for outsourcing, which it most definitely is not," Huston says. "Outsourcing is when I hire someone to perform a service and they do it and that's the end of the relationship. That's not much different from the way employment has worked throughout the ages. We're talking about bringing people in from outside and involving them in this broadly creative, collaborative process. That's a whole new paradigm."

4. The Masses

In the late 1760s, a Hungarian nobleman named Wolfgang von Kempelen built the first machine capable of beating a human at chess. Called the Turk, von Kempelen's automaton consisted of a small wooden cabinet, a chessboard, and the torso of a turbaned mannequin. The Turk toured Europe to great acclaim, even besting such luminaries as Benjamin Franklin and Napoleon. It was, of course, a hoax. The cabinet hid a flesh-and-blood chess master. The Turk was a fancy-looking piece of technology that was really powered by human intelligence. Which explains why Amazon.com has named its new crowdsourcing engine after von Kempelen's contraption. Amazon Mechanical Turk is a Web-based marketplace that helps companies find people to perform tasks computers are generally lousy at --identifying items in a photograph, skimming real estate documents to find identifying information, writing short product descriptions, transcribing podcasts. Amazon calls the tasks HITs (human intelligence tasks); they're designed to require very little time, and consequently they offer very little compensation --most from a few cents to a few dollars.

InnoCentive and iStockphoto are labor markets for specialized talents, but just about anyone possessing basic literacy can find something to do on Mechanical Turk. It's crowdsourcing for the masses. So far, the program has a mixed track record: After an initial burst of activity, the amount of work available from requesters --companies offering work on the site --has dropped significantly. "It's gotten a little gimpy," says Alan Hatcher, founder of Turker Nation, a community forum. "No one's come up with the killer app yet." And not all of the Turkers are human: Some would-be workers use software as a shortcut to complete the tasks, but the quality suffers. "I think half of the people signed up are trying to pull a scam," says one requester who asked not to be identified. "There really needs to be a way to kick people off the island."

Peter Cohen, the program's director, acknowledges that Mechanical Turk, launched in beta in November, is a work in progress. (Amazon refuses to give a date for its official launch.) "This is a very new idea, and it's going to take some time for people to wrap their heads around it," Cohen says. "We're at the tippy-top of the iceberg."

A few companies, however, are already taking full advantage of the Turkers. Sunny Gupta runs a software company called iConclude just outside Seattle. The firm creates programs that streamline tech support tasks for large companies, like Alaska Airlines. The basic unit of iConclude's product is the repair flow, a set of steps a tech support worker should take to resolve a problem.

Most problems that iConclude's software addresses aren't complicated or time-consuming, Gupta explains. But only people with experience in Java and Microsoft systems have the knowledge required to write these repair flows. Finding and hiring them is a big and expensive challenge. "We had been outsourcing the writing of our repair flows to a firm in Boise, Idaho," he says from a small office overlooking a Tully's Coffee. "We were paying $2,000 for each one."

As soon as Gupta heard about Mechanical Turk, he suspected he could use it to find people with the sort of tech support background he needed. After a couple of test runs, iConclude was able to identify about 80 qualified Turkers, all of whom were eager to work on iConclude's HITs. "Two of them had quit their jobs to raise their kids," Gupta says. "They might have been making six figures in their previous lives, but now they were happy just to put their skills to some use."

Gupta turns his laptop around to show me a flowchart on his screen. "This is what we were paying $2,000 for. But this one," he says, "was authored by one of our Turkers." I ask how much he paid. His answer: "Five dollars." Contributing editor Jeff Howe (jeff_howe@wiredmag.com) wrote about MySpace