On 'Jeopardy!' It's man vs. this machine

February 13, 2011

Nell Greenfieldboyce

Download Story

The greatest Jeopardy! champions of all time are returning to the TV screen, and this time they're not playing just for money — they're playing for all of humanity.

That's because they'll be competing against Watson, a computer system built by IBM to prove that machines can master the kind of tricky human language featured on the quiz show, where confusing clues often involve puns, jokes and wordplay.

Computer experts say this competition is the "natural language processing" equivalent of the 1997 chess match between IBM's Deep Blue supercomputer and world chess champion Garry Kasparov.

Back then, some people downplayed the Deep Blue's victory, saying that chess basically all boils down to math — so it's right up a computer's alley, says Oren Etzioni, director of the Turing Center at the University of Washington in Seattle.

"People said, 'OK, that's amazing, but ultimately, look, this is chess, this is something that is very precise, it is very constrained,' — black and white, if you will," Etzioni says.

Jeopardy! is a different story. The game is full of the kind of playful human language that has traditionally baffled literal-minded computers. A computer that converses naturally with people has long been an elusive goal for artificial intelligence researchers.

That's what made Jeopardy! such an enticing challenge for IBM. About two dozen of its researchers spent four years building Watson. David Ferrucci, the team's leader, says that while figuring the answer to Jeopardy! questions can be hard, the first hurdle for the computer is figuring out the question itself — something that's effortless for a human.

"I mean, the computer has to find out, you know, where are the individual words and then how do the words group together," Ferrucci says. "What's the verb? What's the subject? What's the object? What's the preposition? What's the object of the preposition?"

Even after doing that, he says, Watson still doesn't understand a word's meaning in the same way that a human with real-world experience can.

"When we hear language, we bring so much context to interpreting the question that we come up with sensible and reasonable answers," Ferrucci says. "The computer struggles with that."

How Watson Works

To aid in this struggle, Watson's creators gave it awesome speed and memory. It searches through some 200 million pages of reference material — everything from the Bible to encyclopedias to novels — looking for the words or phrases in the Jeopardy! clue and seeing what other words are frequently associated with them on these pages. This produces a lot of potential answers. But which one is most likely to be correct?

"It reads other things and says, does this passage support this as the answer?" Ferrucci explains. Watson simultaneously performs many different forms of analysis and weighs the evidence using experience it previously gained by studying thousands of Jeopardy! clues plus their correct solutions, he says.

In mere seconds, Watson comes up with a list of possible answers and ranks them. The top choice is the one it has the most confidence in, and if that confidence reaches a certain threshold, it buzzes in. This fall, the computer system played dozens of sparring matches against former Jeopardy! Tournament of Champions contestants to get ready for its big day.

Watson does make mistakes, and Ferrucci says the errors can be funny and revealing.

"One of my favorites is, 'What do grasshoppers eat?' and it came back and said 'kosher,' " Ferrucci says, noting that grasshoppers are apparently a kosher food and Watson made connections between the two words without comprehending the real point of the question.

What's The Future Of Artificial Intelligence?

Artificial intelligence experts say that even with the occasional error, Watson has what it takes to perform very well against humanity's best. Etzioni expects to see a Watson win in the competition, which will air Feb. 14, 15 and 16.

"Does that mean that it's 'Game Over' for humans, that robots will keep us as pets? Absolutely not," Etzioni says. "But it does mean it's a demonstration that we've significantly expanded the envelope of what computers can really achieve."

That view is seconded by Henry Kautz, chairman of the department of computer science at the University of Rochester and president of the Association for the Advancement of Artificial Intelligence, who says an answering machine like Watson really fulfills a kind of sci-fi vision of what a computer should be.

"You ask it a question, it gives you the answer," Kautz says. "It doesn't give you 100 pages of possible answers."

He thinks that even if Watson doesn't think exactly like a human, it's not totally alien, either, because the human brain is constantly taking in clues about the world and trying to make associations. "There's no contradiction between saying that Watson just looks at statistical associations and saying it's intelligent," Kautz says, "because I do think that a lot of human intelligence is the same way." Copyright 2011 National Public Radio. To see more, visit http://www.npr.org/.