IBM WATSON

Watson and Jeopardy!

Abstract
What did IBM and Jeopardy! announce on 4/27? IBM unveiled details of an advanced computing system that will be able to compete with humans at the game of Jeopardy!. Additionally, officials from Jeopardy! announced plans with IBM to produce a human vs. machine contest on the renowned quiz show. show's only human-versus-machine matchup.In a two-game, combined-point match, broadcast in three Jeopardy! episodes February 1416, Watson bested Brad Rutter, the biggest all-time money winner on Jeopardy!, and Ken Jennings, the record holder for the longest championship streak.Watson received the first prize of $1 million, while Ken Jennings and Brad Rutter received $300,000 and $200,000, respectively. Jennings and Rutter pledged to donate half their winnings to charity, while IBM divided Watson's winnings between two charities. Watson consistently outperformed its human opponents on the game's signaling device, but had trouble responding to a few categories, notably those having short clues containing only a few words. For each clue, Watson's three most probable responses were displayed by the television screen. Watson had access to 200 million pages of structured and unstructured content consuming four terabytes of disk storage, including the full text of Wikipedia. Watson was not connected to the Internet during the game

For nearly two years IBM scientists have been working on a highly advanced Question Answering (QA) system, codenamed "Watson." The scientists believe that the computing system will be able to understand complex questions and answer with enough precision, confidence, and speed to compete on Jeopardy! Watson is an artificial intelligence computer system capable of answering questions posed in natural language,developed in IBM's DeepQA project by a research team led by principal investigator David Ferrucci. Watson was named for IBM's first president, Thomas J. Watson. In 2011, as a test of its abilities, Watson competed on the quiz show Jeopardy!, in the

What powers Watson?

Watson is the only computer on the planet that can answer a Jeopardy! question in less than three seconds - fast enough to be competitive with the world s best human players. Curious how it answers so fast? How much information it has stored? How it is built? Watson in the world The Deep QA technology that powers Watson could soon power many solutions in our world. In fact, IBM is already working to implement applications in the fields of healthcare, finance and telecom.

On stage

Figure 1: Avatar: This avatar is wired to react to inputs like Watson s level of processing changing color, for example, to indicate its confidence in a response and whether it answers correctly or incorrectly. Answer panel: Watson s top three responses to each clue are displayed in an answer panel, along with its confidence level. If the top response exceeds the confidence threshold, Watson will buzz in and answer.

Buzzer : Watson uses the same hand buzzer as any other Jeopardy! contestant. Instead of using a hand to buzz, Watson is equipped with a mechanical device to press the button.

3 seconds: The Watson team discovered that on average, a champion-level Jeopardy! contestant buzzes in 3.5 seconds after a clue is revealed. Watson averages less than 3 seconds per clue, the final speed for the competition. The clues: Since Watson cannot see or hear, the system receives each clue digitally as it is revealed. Then, Watson s thousands of algorithms begin to analyze the clue and search through its troves of natural language documents for answers.

system
90 servers: The system powering Watson consists of 10 server racks and 90 IBM Power 750 servers based on the POWER7 processor. 2 hours: A computer with a single processor core takes more than 2 hours to perform the deep analytics needed to answer a single Jeopardy! clue. Watson does this in less than three seconds. 75 % : During its series of preliminary sparring matches, Watson was only utilizing about 75% of its total processing resources 2,880 processor cores: The computing power of Watson can be compared to over 2,880 computers with a single processor core, linked together in super high-speed networks.

Servers

Applications: The POWER7 processor inside the Power 750 is designed to handle both computation-intensive and transactional processing applications from weather simulations, to banking systems, to competing against humans on Jeopardy! 100X: The POWER7 processor inside the Power 750 is designed to handle both computation-intensive and transactional processing applications from weather simulations, to banking systems, to competing against humans on Jeopardy! Bytes Vs Brains: Each Power 750 server measures 6.9 x 17.3 x 28.7 and weighs 120 lbs. The average human brain measures 3.7 x 5.5 x 6.6 and weighs around 3.3 lbs.

Architecture
10x10x10x10 : Watson uses Apache UIMA to scale out its natural language processing in parallel across its POWER7 processors, allowing Watson to perform thousands of analytical computations simultaneously across the server cluster to answer each question as fast as possible. UIMA : Watson is designed according to Unstructured Information Management Architecture UIMA for short. This software architecture is the standard for developing programs that analyze unstructured information such as text, audio and images. System fact : IBM collaborated with several other companies to create the Unstructured Information Management Architecture standard. The UIMA code was then donated to the Apache Software Foundation, where it is now open source and available for everyone

Workload
Watson is optimized to answer each question as fast as possible. The same system could also be optimized to answer thousands of questions in the shortest time possible. This scalability is what makes Watson so appealing for business applications.

Energy
2,880 cores : In the past, the way to speed up processing was to speed up the processor. This consumed more energy and generated more heat. Watson scales its computations over 90 servers, each with 32 POWER7 cores running at 3.55 GHz. This provides greater performance and consumes less power. What s few watts?? Delivering the deep analytics Watson provides in less than three seconds takes less energy than you would think. Watson consumes fewer kilowatts of power during a typical Jeopardy! 5,000 MW In 2005, data centers in the U.S. consumed about 5,000 megawatts of power. That is roughly the equivalent yearly output of five typical power plants.

Network
9000x The network that connects the system s servers can handle 90 x 10 billion bits per second. In contrast, a typical Ethernet network for a home is rated at 100 Mb/s, or 90,00 times slower than Watson. Speed + smarts Watson generates hundreds of possible answers, evaluates each simultaneously and narrows its responses down to its top choice in about the same time it takes a human champion to come up with their answer. Speed counts in a system doing this amount of processing in such little time. 10Gb speed Watson is not connected to the Internet. However, the system s servers are wired together by a 10 Gigabit Ethernet network.

Cooling
64- F The two 20-ton air conditioning units that regulate the temperature of the Watson server room are enough to cool a room about one-third the size of a football field. Data center facts Up to 50% of the energy consumed by a modern data center goes toward air cooling. 2x20 tons A Jeopardy! match can get a little hot, so Watson needs a way to cool down. The Watson server room is cooled by two industrial grade, 20-ton air conditioning units.

Storage
Is that all?? 500 GB might not seem like enough knowledge to compete on the quiz show Jeopardy!. But consider this: Watson mainly stores natural language documents which require far less storage than the image, video and audio files on a personal computer. Tera-brains : Theoretical estimates of the human brain s memory capacity vary between one and 1,000 terabytes. 500Gigabytes : 500 gigabytes of disk hold all the information Watson needs to compete on Jeopardy!. This data size is equivalent to about 200 million printed pages of text.

What kind of technology is Watson based on? Watson is an application of advanced Natural Language Processing, Information Retrieval, Knowledge Representation and Reasoning, and Machine Learning technologies to the field of open-domain question answering. At its core, Watson is built on IBM's DeepQA technology for hypothesis generation, massive evidence gathering, analysis, and scoring. What is the difference between Watson and IBM's overall QA research? Watson, besides being the name of the founder of IBM and the name of the IBM T.J. Watson Research Laboratory, is the name of the computer system that will play Jeopardy! against humans in the planned contest. IBM's DeepQA research project and question answering technology are being used to create the Watson computer system. What does it take to win at Jeopardy!? Jeopardy! is a game covering a broad range of topics, such as history, literature, politics, arts and entertainment, and science. Jeopardy! poses a grand challenge for a computing system due to its broad range of subject matter, the speed at which contestants must provide accurate responses, and because the clues given to contestants involve analyzing subtle meaning, irony, riddles, and other complexities in which humans excel and computers traditionally do not. To win, it takes the deep analysis of large volumes of content to deliver high accuracy, confidence and speed. The best Jeopardy! players, according to our analysis, provide correct, precise responses more than 85% of the time. They also "know what they don't know" and choose not to answer questions when they are unsure, since there is a penalty for being wrong. How Good is Watson? Watson will not be able to answer every possible Jeopardy! question. However, IBM has established a promising approach and is developing a robust and fast implementation. We expect to reach competitive levels of human performance but have a ways to go and will be partnering with willing universities to push the envelope.

Is web search enough to win Jeopardy!? A web search engine has access to an immense source of information and can quickly find relevant web pages given a small number of query terms. To play Jeopardy!, however, you must generate precise answers to the questions. A web search engine doesn't return precise answers; rather, it is designed to return a ranked list of web pages the user may be trying to find. To answer a Jeopardy! clue you must go beyond the search result page, dig into documents that are likely to contain the answer, fetch them, read them, and locate the precise and correct answer within them. A challenge for even finding the relevant documents is the issue of choosing the right set of keywords to retrieve those documents in the first place. Many Jeopardy! clues contain information that is not critical for answering the clue but is provided for educational and/or entertainment purposes. Moreover, the clues may use terms different from those used in answer

bearing documents. As a result, formulating an effective query that homes in on the relevant documents is a critical and non-trivial task. Even if you think you've found an answer to the question, there's still the issue of confidence. Remember, you are penalized for giving a wrong answer, so you must have enough confidence in your answer to decide you even want to attempt answering the question. To obtain this confidence you need to read enough supporting text around the answer to convince yourself that you have found the correct answer. You may even want to see the answer justified by multiple sources, especially if the penalty for getting it wrong is significant. And then there are the clues with answers that must be synthesized the answer is a list of items or a logical combination of two or more items. These answers do not appear in any one place. Rather, you must synthesize them from independent sources to form your final answer. On top of all this you must be ready with a confident answer in a matter of seconds after receiving the clue. The bottom line is that the Jeopardy! Challenge poses a different kind of problem than what is solved by web search. It demands that the computer deeply analyze the question to figure out exactly what is being asked, deeply analyze the available content to extract precise answers, and quickly compute a reliable confidence in light of whatever supporting or refuting information it finds. IBM believes that an effective and general solution to this challenge can help drive the broader impact of automatic question answering in science and the enterprise. What data is stored in Watson? All of Watson's data will be self-contained. Watson will perform without a connection to the web or any external resource. The vast majority of Watson's data will be a wide variety of natural language text. Some structured (formal knowledgebase's) and semi-structured data (tagged text) is also included mostly to help interpret text and refine answers. Exactly which data will be used for competing on Jeopardy! will be revealed at a later date, but the specific content and how to analyze and manage it are part of the research agenda. How can you find all these answers without being connected to the Internet? Watson will not have enough data to answer every possible Jeopardy! question in its selfcontained memory, nor can it possibly predict the questions it will get. In this sense it has the same limitations as do the best human contestants. The entire Watson computer system will be self-contained and on stage as are the human contestants no external connections, no life-lines what you see is what you get. The purpose of this technology showcase is to demonstrate the system's ability to deeply analyze the data it does have and to compute accurate confidences based on supporting or refuting natural language evidence. Think of it as if Watson has read a lot of books and in real time relates what it read to the question to find and support the right answers.

Will Watson be able to answer Audio/Visual clues? While IBM is developing a host of advanced technologies for audio, image and video analysis, the first time around we are focusing squarely on the natural language understanding challenge. Future applications of the technology are planned to incorporate audio, image and video capabilities. How can Watson handle the puns and wordplay that occur in Jeopardy? By reading many, many texts Watson learns how language is used. That means it learns the context and associations of words, which allows it to deal with some of the wordplay we find in Jeopardy!. But what is special about Watson is that it is able to also produce a confidence in its answer. If that confidence is low, it realizes: maybe it doesn't understand the question--maybe there is a pun or something it's not getting. On the other hand, if the confidence is high, it knows it likely understood the question and stands a good chance of getting that question right. When will the contest take place on Jeopardy! and who specifically will Watson play against? The first step is for Watson to demonstrate its worthiness to play against champions by competing in a series of sparring matches starting sometime this year. The date and specific contestants for the final match have not been decided. When the date is set and the contestants are decided, Jeopardy! and IBM will make a public announcement. On what kind of computer does this run? To achieve the levels of accuracy, confidence, and speed required by the Jeopardy! Challenge, a massively parallel high performance computing platform, like BlueGene maybe used. The system can be scaled up or down depending on different application requirements. Why should it surprise me that a computer might beat a human at Jeopardy!? Can't supercomputers do this today? While computers can store and deliver a wealth of digital content created by humans, they are unable to operate over it in human terms. The quest for building a computer system that can do open-domain Question Answering is ultimately driven by a broader vision that sees computers operating more naturally in human terms rather than strictly computer terms. They should function in ways that understand complex information requirements, as people would express them, for example, in natural language questions or interactive human dialogs. Computers should deliver precise, meaningful responses, and synthesize, integrate, and rapidly reason over the breadth of human knowledge as it is most rapidly and naturally produced in natural language text. While competing at Jeopardy! is not the end-goal, it is a milestone in demonstrating a capability that no computer today exhibits the ability to interact with humans in human terms over broad domains of knowledge.

Can other computer systems do this now? Why doesn't IBM take on another company's computer in this competition? It seems like that would be more challenging than taking on human contestants. According to our analysis, two competing champion Jeopardy! players combined are able to answer 85%-90% of the questions over an incredibly broad domain of topics and accurately predict their own correctness with very high levels of confidence. And they do that in just seconds. IBM has not witnessed that level of performance by any other computer system to date. While other companies may be working on similar technologies, at this stage of the game the best human Jeopardy! players are the folks to beat.

How is IBM working with Universities in the general field of Question Answering? Early last year (2008), IBM and Carnegie Mellon University, along with several other universities doing research in this space, kicked-off the "The Open Advancement of Question Answering" (OAQA) initiative. This broad collaboration is intended to provide a foundation for effective collaboration among researchers to accelerate the science of automatic question answering. The initiative aims to develop common metrics, architectures, experimental methodologies, tools and driving challenge problems, like the Jeopardy! Challenge, to facilitate the collaborative and rapid advancement of the state-of-the-art in QA. IBM intends to invite interested universities to collaborate on learning how to integrate their advanced QA component technologies and apply the key approaches in DeepQA (the technology underlying Watson) to different problems based on OAQA methodologies. Does DeepQA use UIMA? Yes. UIMA is a standard framework for building applications that perform deep analysis on unstructured content, including natural language text, speech, images and video. IBM contributed UIMA to open-source (see the Apache UIMA web site) to help facilitate and accelerate work in deep content analytics. UIMA is also now an OASIS standard. UIMA-AS implements UIMA on asynchronous messaging middleware. DeepQA and the Watson system uses UIMA-AS as its principal infrastructure for assembling, scaling-out and deploying all its analytic components

Conclusion
Healthcare Medical records, texts, journals and research documents are all written in natural language a language that computers traditionally struggle to understand. A system that instantly delivers a single, precise answer from these documents could transform the healthcare industry. Finance Enormous amounts of data are generated every day in the financial industry. Watson, the IBM computer system designed to compete on Jeopardy!, has the deep analytics capability that could help businesses extract knowledge from this data in order to identify patterns and make more informed financial decisions. Customer Services

IBM experts share their thoughts on how DeepQA technology could help transform the customer service industry into a faster, more accurate experience. Imagine if you had a system where you just called Watson, asked a question, and you get the answer in real time.

References y IBM Watson: The Face of Watson y The DeepQA Project y IBM's "Watson" Computing System to Challenge All Time Greatest Jeopardy! Champions y Zimmer, Ben (2011-02-17), Is It Time to Welcome Our New Computer Overlords? y Google y www.uima.apache.org y www.ibmwatson.com