IBM Watson Jeopardy

[huskiesgoaler34]

Truly an unbelieveable piece of programming and AI. Just want to hear some of your opinions on if Watson is the smartest machine on Earth and if. or when he will be used in society.

[andrew.]

I don't know very much about this kind of stuff, but to me it seems like Watson basically just analyzes phrases for keywords and simply searches for it in it's database. In fact, I was reading an article about Watson and the first "prototype," if you could even call it that, basically looked for keywords and then typed it in Google to find Wiki pages about the topic.

[Insectoid]

According to Wikipedia, Watson has 4 terabytes of data on hard drives, and still manages to answer in under 3 seconds. That's pretty damn impressive.

[Brightguy]

Smartest machine? Perhaps, but only in a limited domain. Watson-like systems will probably start to become more standard over the next 5–15 years. I wonder if IBM would allow Watson to take queries over the Internet à la Wolfram.

Random thoughts:
• The specific categories used made all the difference. It happened to be an expert in hedgehogs and études, but clueless about keyboards and cheap things.
• It has very fast reflexes (which ultimately probably won it the game) but it seems to think slower than humans. It got shut out in the "actors who direct" category simply because the clues were so short, and Trebek could finish reading them quickly.
• It got lucky and landed on 5 of the 6 daily doubles, even though it was only controlling the board about half the time.
• It had trouble determining the proper category. A lot of the responses it got wrong were hilarious. "Dorothy Parker" is not a book, "leg" is not an anatomical oddity, "chemise" is not on a keyboard, and "Picasso" isn't an art period (and it was 97% sure of that one). And of course...

Spoiler:

Toronto is not even in the United States

[Dan]

Brightguy @ 19th February 2011, 1:53 am wrote:

? It got lucky and landed on 5 of the 6 daily doubles, even though it was only controlling the board about half the time.

Daily doubles are not placed comptely randomly on the board. Their are spaces that are far more likely to contain one, and it has been a valid jeopardy strategy to try and hunt for them. This is what watson was doing during the rounds that cointated them and part of the reason why it would jump between categories and start in seemling odd places in them.

[Brightguy]

Dan @ Sat Feb 19, 2011 4:31 am wrote:

Daily doubles are not placed comptely randomly on the board. Their are spaces that are far more likely to contain one, and it has been a valid jeopardy strategy to try and hunt for them. This is what watson was doing during the rounds that cointated them and part of the reason why it would jump between categories and start in seemling odd places in them.

Except the humans were also doing that, so it still was lucky in that regard. In the final round the last daily double was probably in the third or fourth row, and Ken guessed wrong with two clues remaining. If he had of guessed right and answered correctly, Watson would not have had the tournament lock going into Final Jeopardy (but would've still won).

Insectoid @ Fri Feb 18, 2011 11:23 pm wrote:

According to Wikipedia, Watson has 4 terabytes of data on hard drives, and still manages to answer in under 3 seconds. That's pretty damn impressive.

I think that was referring to the training data. It doesn't use hard drives while playing because that would be too slow. Who needs hard drives when you have 16 terabytes of RAM?

[Insectoid]

yeah, I misread that. The 4 TB was from an older version of Watson.

[Sur_real]

when AIs start to learn from their mistakes and adapt, that's when I'll be really impressed (and even scared). I remember times in the game when either Ken or Brad (I forgot) answered a question wrong, Watson chose to repeat the wrong answer and therefore not learning from his mistakes.
I'm not saying Watson's not smart, he is, but that's only because of his processing power/ability and not because he can mimic humans (the ability to reason, self-awarness, etc).
This reminds me...I wonder if Watson can host a full crysis tournament...that'll be cool

[ProgrammingFun]

Sur_real @ Sat Feb 19, 2011 6:09 pm wrote:

I wonder if Watson can host a full crysis tournament...that'll be cool

We might never know...unless Crytek and IBM are good friends

[Tony]

Sur_real @ Sat Feb 19, 2011 6:09 pm wrote:

Watson chose to repeat the wrong answer and therefore not learning from his mistakes.

Watson does learn from its own mistakes, in a way that it might take a few questions for it to become more confident in a new category. It takes no audio input, so it can't be aware of wrong answers from other players.

[Sur_real]

Tony @ Sat Feb 19, 2011 6:30 pm wrote:

Sur_real @ Sat Feb 19, 2011 6:09 pm wrote:

Watson chose to repeat the wrong answer and therefore not learning from his mistakes.

Watson does learn from its own mistakes, in a way that it might take a few questions for it to become more confident in a new category. It takes no audio input, so it can't be aware of wrong answers from other players.

yeah, that's true and he does have an algorithm to calculate how much to wager but I guess you pointed out something else about Watson--not being able to do audio inputs and therefore be on par with humans in terms of how to interpret the information.
On a related note, how does Watson know Alex finished reading out the sentence since they are only allowed to buzz after he finished reading right? That's why I thought Watson had at least some kind of voice recognition. Also, on past jeopardys, there were audio and video daily doubles, I didn't watch the whole thing so I wonder if there were any of those vs Watson?

[Alexmula]

Sur_real @ Sun Feb 20, 2011 11:52 am wrote:

Tony @ Sat Feb 19, 2011 6:30 pm wrote:

Sur_real @ Sat Feb 19, 2011 6:09 pm wrote:

Watson chose to repeat the wrong answer and therefore not learning from his mistakes.

Watson does learn from its own mistakes, in a way that it might take a few questions for it to become more confident in a new category. It takes no audio input, so it can't be aware of wrong answers from other players.

yeah, that's true and he does have an algorithm to calculate how much to wager but I guess you pointed out something else about Watson--not being able to do audio inputs and therefore be on par with humans in terms of how to interpret the information.
On a related note, how does Watson know Alex finished reading out the sentence since they are only allowed to buzz after he finished reading right? That's why I thought Watson had at least some kind of voice recognition. Also, on past jeopardys, there were audio and video daily doubles, I didn't watch the whole thing so I wonder if there were any of those vs Watson?

watson probably estimates the amount of time it takes alex to read the question based on the length of the question and how fast alex talks

[Tony]

I'm not familiar with the exact Jeopardy rules, but from reading a Q&A with Ken about the game, a valid aggressive strategy was to buzz in _before_ the clue was read, finish reading the clue and hope that you know the answer, all within the allowed buffer time (5 seconds?). The strategy is viable when you are playing at this level, feel confident in a category, and the clues are short.

Otherwise Watson certainly has a buzzer advantage.

[Brightguy]

Tony @ Sun Feb 20, 2011 4:42 pm wrote:

I'm not familiar with the exact Jeopardy rules, but from reading a Q&A with Ken about the game, a valid aggressive strategy was to buzz in _before_ the clue was read, finish reading the clue and hope that you know the answer, all within the allowed buffer time (5 seconds?). The strategy is viable when you are playing at this level, feel confident in a category, and the clues are short.

Not quite. If you buzz in before Trebek finishes reading you can't buzz in for another 0.25 seconds. The humans are alerted by a studio light when Trebek is finished talking, and Watson is alerted by an electronic signal. If Watson is confident enough, it buzzes in within 0.004 seconds. That is too fast for a human to beat if they wait to react to the light, so to beat Watson you have to time your buzzer press exactly as Trebek is finishing and just hope you don't get locked out.

Sur_real @ Sun Feb 20, 2011 11:52 am wrote:

Also, on past jeopardys, there were audio and video daily doubles, I didn't watch the whole thing so I wonder if there were any of those vs Watson?

Audio and video daily doubles were excluded from this match because of Watson's limitations.