While many of us may be impressed at the trouncing IBM’s supercomputer Watson gave to "Jeopardy!" champs Ken Jennings and Brad Rutter this week, the director of the Government Computer News (GCN) labs was unimpressed.
Writing today, Feb. 17, 2011 on GCN’s website, John Breeden II said, "You would think that a techie like me would be overjoyed at seeing a glimpse into the rise [of] what show loser Ken Jennings called [referring to a ‘Simpsons’ episode] ‘our new computer overlords.’ But actually I was disgusted by the whole dog and pony show, which was set up in a blatantly unfair way to favor the computer."
That’s because, Breeden writes, "Watson had an unfair advantage: It was being fed the questions electronically. I wanted to see Watson hear the questions using speech recognition and process them the way humans do. But Watson was instead fed the words that made up the question in ASCII text and then went about searching a database, albeit a good one, looking for patterns and coming up with the proper response. All very rudimentary work for a computer, actually, and not much different than what Google and Bing do every day right now."
Jennings and Rutter played Watson for two games. After the final game, televised on Wednesday night, Feb. 16, 2011, the tally was $77,147 for Watson, $24,000 for Ken Jennings and $21,600 for Brad Rutter.
According to a story in The New York Times, IBM researchers acknowledged that Watson benefited from the "buzzer factor," because its human competitors had to anticipate the light that signals it’s possible to "buzz in" to provide a response, while the computer uses a weighted scheme that allows it to buzz in as quickly as 10 milliseconds. That made it hard for the human competitors to hit their buzzers in time.
Furthermore, Breeden writes, "The fact that Watson had a buzzing device is irrelevant. It already knew how it would answer before the question was finished being read, and the humans were still gathering input. And considering that the questions on last night’s show were actually pretty easy for “Jeopardy!” and that Jennings and Rutter obviously knew most of the answers, what Watson really excelled at was buzzing in faster than the humans."
We here at TVWeek noted that sometimes, when a answer was written in a way more easily understood by humans, Watson had trouble. For example, the Final Jeopardy answer for game one, in the category U.S. Cities, was: "Its largest airport is named for a World War II hero; its second largest, for a World War II battle." Both Jennings and Rutter got the questions right–What is Chicago (which has O’Hare and Midway airports). Watson was completely lost and came up with "What is Toronto."
The WGBH PBS science show "Nova"aired an excellent program this month about IBM’s Watson project, and it’s well worth watching: