Using HTTP Adapters to access Watson Question and Answer service at IBM Bluemix

Important: IBM Watson Question and Answer service has been disconinued. See https://developer.ibm.com/watson/blog/2015/11/11/watson-question-and-answer-service-to-be-withdrawn/
Download Studio project or fork it on Github

Introduction

The goal of this article is to present how to create adapters that communicate with the Watson Question and Answer service available in IBM Bluemix.

Bluemix is the PaaS (Platform as a Service) environment from IBM that also provides a wide variety of micro services for developers and software development companies.

Watson Question and Answer (WQ&A for now on) is a powerful Q&A service powered by the IBM Watson cognitive system and available at IBM Bluemix platform through a REST interface. A live demo of the WQ&A capabilities is available at http://watson-qa-demo.mybluemix.net/

With the adapters presented in this article, it is possible to create cognitive Q&A apps that uses natural language as the input.

A Sample project with a hybrid app is also provided in this article to exemplify the usage of the proposed adapters but the details of the mobile app are omitted here to keep the focus on the server side (adapters).

WQ&A is currently a Beta service and has only two knowledge domains (corpus) available - travel and healthcare - but it will increase with time and maturity of the service.

We will use a simple HTTP Adapter to connect to WQ&A API.

Watson Question and Answer API

The details of the service are available from Bluemix documentation at

http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/question-answer.html

The operations and endpoints available from the API are the following:

GET - ping - path: /v1/ping

The ping endpoint returns the status of the service. The response is in the form of HTTP Status Code. Status 200 indicates that the service is properly available.

GET - services - path: /v1/services

The services endpoint lists the available corpus, the types of content/questions that the service is trained to answer. Initially the contents available are travel and healthcare so the service can answer travel questions like "How much should I tip a taxi in Argentina?" or healthcare questions like "What is HIV?"

It's important to always get the services list dynamically as the corpus can increase with new knowledge areas available. The response obtained is in the format of a json document like this:

1
{"services": ["travel","healthcare"]}

POST - question - path: /v1/question/{id}

The question endpoint is used to send a question to be responded by Watson. The question is sent as a json document with several optional parameters. The following is an example of json document with a question using only the required parameters.

1
2
3
4
5
{
"question": {
   "questionText":"What is HIV?"
    }
}

The response is in the form of a json document containing:

  • a list of answers with the confidence level of each answer
  • the evidence list that support the given answers
  • a set of metadata about the responses and evidence list obtained

Following is an example of the response obtained:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
   "array": [
  	{
     	"question": {
        	"answers": [
           	{
                  "confidence": 0.46911,
                  "id": 0,
                  "pipeline": "Descriptive,TAO",
                  "text": "472D471A96B201B99D8F1C200E7E242B - Get Tested for HIV : Get Tested for HIV : The Basics : What is HIV?"
               },
           	<other answers...>
        	],
        	"category": "",
        	"errorNotifications": [
        	],
        	"evidenceRequest": {
           	"items": -1,
           	"profile": "NO"
        	},
        	"evidencelist": [
           	{
                  "copyright": "No copyright specified for this supporting passage or document.",
                  "document": "http:\/\/10.110.76.161:8080\/instance\/67\/deepqa\/v1\/question\/document\/PB_472D471A96B201B91949C3B1C28D468B\/175\/256",
                  "id": "PB_472D471A96B201B91949C3B1C28D468B",
                  "metadataMap": {
                     "DOCNO": "472D471A96B201B91949C3B1C28D468B",
                     "abstract": "Protect yourself and others from HIV and other STDs.",
                     "corpusName": "PB",
                     "deepqaid": "33598696",
                     "description": "Protect yourself and others from HIV and other STDs.",
                     "fileName": "\/watsondata\/ingestion\/201410032203\/eng\/di\/output\/trec\/intermediate\/xml-splitTrecTrim\/33598696.xml",
                     "originalfile": "Get Tested for HIV.html",
                     "title": "Get Tested for HIV : Get Tested for HIV : The Basics : What is HIV?"
                  },
                  "termsOfUse": "No license specified for this supporting passage or document.",
                  "text": "HIV stands for human immunodeficiency virus. This is the virus that causes AIDS.",
                  "title": "Get Tested for HIV : Get Tested for HIV : The Basics : What is HIV?",
                  "value": "0.46911129355430603"
               },
           	< other evidence....>
        	],
        	{ OTHER METADATA }
     	}
  	}
   ]
}

PUT - feedback - path: /v1/feedback

The feedback endpoint is used to provide the user's feedback to Watson regarding the precision of the answers received. This information is used by Watson to continuously learn and improve next answers according the perception of end user (or someone that is actually teaching Watson in a new area). Feedback is provided as a json document containning the informations about the original question, one of the received answers and the feedback, in the form of a number: -1,0,1 or 9, according to this rule:

  • -1=answers was irrelevant
  • 0=neutral feedback
  • 1=answer was relevant
  • 9=answer was partially relevant
1
2
3
4
5
6
7
8
9
10
  "questionId": "1",
  "questionText": "What are the symptoms of diabetes?",
  "answerId": "0",
  "answerText": "Some people who have diabetic heart disease (DHD) may have no signs or symptoms of heart disease.",
  "confidence": "0.42134",
  "shown": true,
  "evidenceViewed": true,
  "feedback": "-1",
  "comment": "This is not a good answer"
}

Lets see how to access the REST API with HTTP Adapters:

Part 1 - Access and configure IBM Bluemix services

To access the WQ&A service it is required to have an account in bluemix

Create your account in Bluemix

  1. Access bluemix.net and create an account in the SIGN UP option.
  2. missing_alt

    Create a new bluemix application

  3. After accessing Bluemix, select the option "CREATE AN APP"to create a new Bluemix App
  4. missing_alt
  5. Select the option to create a Runtime "Liberty for Java". A Runtime in Bluemix is an application server that will run on Bluemix platform. We will not actually use a Runtime in this app but it is required it in order to access your Watson credentials (all services in Bluemix must be attached to a Runtime of any type).
  6. missing_alt
  7. Give a name to your app(example watson-qa-your-username).
  8. missing_alt
  9. Check the pricing options and select CREATE
  10. Add the Watson Question and Answer service to the app

  11. Click the option "Add a Service" and select the service "Watson Question and Answer"
  12. missing_alt missing_alt
  13. Check the pricing options again and select CREATE
  14. missing_alt
  15. Now you already have a Bluemix App with the Watson Q&A service ready to be used.
  16. Verify the credetials generated to access the WQ&A service clicking at "Show Credentials". A JSON document similar to the following one will be presented.
  17. missing_alt
  18. The important info here are the url, username and password. Those are the credentials to access the WQ&A service (the credentials in the image are just an example). We'll use the credentials in the adapter configuration.
  19. All set on Bluemix side, lets move to MobileFirst Platform

Part 2 - Creating the Adapters to access WQ&A

  1. Open IBM MobileFirst Platform Studio and create a new MobileFirst Project (File > New > MobileFirst Project) and them create a new MobileFirst HTTP Adapter (File > New > MobileFirst Adapter and select HTTP Adapter). Give a name to your adapter like WatsonQA.
  2. missing_alt
  3. Open the .xml configuration file of the adapter (adapters/WatsonQA/WatsonQA.xml)
  4. Change the element according to the following example. Use the values of url, username and password obtained from Bluemix credentials earlier.
  5. 1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    
    <connectionpolicy xsi:type="http:HTTPConnectionPolicyType">
    	<protocol>https</protocol>
    	<domain>gateway.watsonplatform.net</domain>
    	<port>443</port>
    	<authentication>
    		<basic></basic>
    		<serveridentity>
    			<username>1721c34d8-bf3b-4583-8728-a5f10b7be9cc</username>
       			<password>AqUM9nKvUCdr</password>
       		</serveridentity>
       	</authentication>
    </connectionpolicy>
    [/code]
    <h3>Create the procedures to access the WQ&A service</h3>
    <li>Create the procedures to access WQ&A declaring it like the following in the .xml file</li>
    <p>[code lang="xml"]
    <procedure name="ping"></procedure>
    <procedure name="getServices"></procedure>
    <procedure name="postQuestion"></procedure>
    <procedure name="putFeedback"></procedure>
  6. Open the file WatsonQA-impl.js and add the javascript functions ping, getServices, postQuestion e putFeedback according to the following code.
  7. function ping(){
        var input = {
       	 	method : 'get',
       	 	returnedContentType : 'json',
       	 	path : '/qagw/service/v1/ping'
       	 };
        return WL.Server.invokeHttp(input);
    }
    function getServices(){
        var input = {
       	 	method : 'get',
       	 	returnedContentType : 'json',
       	 	path : '/qagw/service/v1/services'
       	 };
        return WL.Server.invokeHttp(input);
    }
    function postQuestion(question_text,type){
        var input = {
       		 method : 'post',
       		 path: '/qagw/service/v1/question/'+type,
       		 returnedContentType: 'json',
       		 body: {
       			 contentType: 'application/json',
       			 content: {
       	   			 question: {
       	  				 questionText:question_text
       				 }
       			 }
       		 }
       	 };
        return WL.Server.invokeHttp(input);
    }
    function putFeedback(_questionId,_questionText,
    _answerId,_answerText,_confidence,_feedback){
        var input = {
       		 method : 'put',
       		 path: '/qagw/service/v1/feedback',
       		 returnedContentType: 'json',
       		 body: {
       			 contentType: 'application/json',
       			 content: {
       				   questionId: _questionId,
       				   questionText: _questionText,
       				   answerId: _answerId,
       				   answerText: _answerText,
       				   confidence: _confidence,
       				   shown: true,
       				   feedback: _feedback
       				 }
       			 }
       	 };
        return WL.Server.invokeHttp(input);
    }
  8. Right click on the adapters folder adapters/WatsonQA and select Run As > Deploy MobileFirst Adapter to deploy the adapter to your development server.
  9. Test your adapter procedures by running it from Run As > Invoke MobileFirst Procedure

Part 3 - Accessing the adapter procedures from a hybrid app

  • To access the procedures from a hybrid app, add the following code to the file apps//common/js/main.js.
  • The functions getWatsonQAServices, postQuestion and putFeedback will access the equivalent adapter procedure and can be used in any ways you want in your application.

    /**
     * getWatsonQAServices will call the procedure getServices to retrieve all the available
     * question domains for the QA services
     * @param onSuccessCallback - Callback function to run on success
     * @param onFailCallback - Callback function to run on fail
     */
    function getWatsonQAServices(onSuccessCallback,onFailCallback){
        var invocationData = {
            	adapter : 'WatsonQA',
            	procedure : 'getServices',
            	parameters : []
        	};
        var options = {
       	 onSuccess : onSuccessCallback,
       	 onFailure : onFailCallback,
        };
        WL.Client.invokeProcedure(invocationData,options);
    }
    /**
     * Call the postQuestion procedure
     * @param question - The question string to be sent. Example: "What are the symptoms of diabetes?- Must be a question in English
     * @param type - A string with the type of question, must be a service obtained with the
     * getWatsonQAServices function. Examples: "healthcare" or "travel
     * @param onSuccessCallback - Callback function to run on success
     * @param onFailCallback - Callback function to run on fail
     */
    function postQuestion(question, type, onSuccessCallback, onFailCallback){
        var invocationData = {
            	adapter : 'WatsonQA',
            	procedure : 'postQuestion',
            	parameters : [question,type]
        	};
        var options = {
       	 onSuccess : onSuccessCallback,
       	 onFailure : onFailCallback,
        };
        WL.Client.invokeProcedure(invocationData,options);
    }
    /**
     * this function receives a feedback index as a number between 0 and 4 and
     * convert feedback to watson feedback standard like explained
     * feedback (string, optional) = ['-1' or '0' or '1' or '9']:
     * String representation of the feedback [-1 = answers was irrelevant,
     * 0=neutral feedback,
     * 1=answer was relevant,
     * 9=answer was partially relevant],
     * @param answerIndex - id of the answer that will receive feedback
     * @param feedbackIndex - Feedback value- must be a number between 0 and 4
     *  @param onSuccessCallback - Callback function to run on success
     * @param onFailCallback - Callback function to run on fail
     */
    function putFeedback(answerIndex,feedbackIndex, onSuccessCallback, onFailCallback){
        var watsonFeedbackIndex=0;
        if(feedbackIndex==1){
       	 watsonFeedbackIndex=-1;//irrelevant
        }
        else if(feedbackIndex==2){
       	 watsonFeedbackIndex=0;//neutral
        }
        else if(feedbackIndex==3){
       	 watsonFeedbackIndex=9;//partially relevant
        }
        else if(feedbackIndex==4){
       	 watsonFeedbackIndex=1;//relevant
        }
        var invocationData = {
            	adapter : 'WatsonQA',
            	procedure : 'putFeedback',
          	//[_questionId,_questionText,_answerId,_answerText,_confidence,_feedback]
            	parameters : [question.id,
                          	question.questionText,
                          	answers[answerIndex].id,
                          	answers[answerIndex].text,
                          	answers[answerIndex].confidence,
                          	watsonFeedbackIndex]
        	};
        //alert("invocation parameters:"+invocationData.parameters);
        var options = {
       	 onSuccess : onSuccessCallback,
       	 onFailure : onFailCallback,
        };
        WL.Client.invokeProcedure(invocationData,options);
    }

    With the functions provided above, you can use the Watson Question and Answer service from any hybrid application.

    Part 4 - Sample application

    Attached to this blog post is a Sample MobileFirst project to access the Watson Q&A service. To load and run the project follow the steps:

    1. Download the project or fork the sample project from github at github.com/leandrodvd/mfp70watsonqa that contains a MobileFirst Platform 7.0 project ready to be imported.
    2. Import the project WatsonQA - File > Import > Existing Projects and select the .zip downloaded
    3. Expand the project folder and open the file adapters/WatsonQA/WatsonQA.xml
    4. Update the values of the parameters and according to the credentials for WQ&A service obtained from Bluemix.
    5. Right click on the folder adapters/WatsonQA and select Run As > Deploy MobileFirst Adapter to deploy the adapter
    6. Right click on the folder apps/WatsonQA and select Run As > Run on MobileFirst Server to deploy the sample app
    7. To preview the app in a browser Right click on the folder apps/WatsonQA/common and select Run As > Preview and the app will load in a browser
    8. Try to send some questions to test Watson skills
    9. The answers and evidence will be presented in the app body along with the confidence level for each answer and the option to provide feedback to Watson is available with the stars (4 stars means a relevant answer and 1 star is for an irrelevant answer)
    10. Remember that Watson Services at Bluemix are in Beta stage so be gentle with Watson if you don't get amazing answers at first, give him some feedback and he will learn fast =)
    11. Following are some screenshots of the app

      missing_alt missing_alt missing_alt missing_alt missing_alt missing_alt

      Conclusion

      In this post it was presented the step by step on how to use a MobileFirst Adapter to access Watson Question and Answer service in Bluemix.

      IBM Bluemix platform contains a wide variety of other services, including other Watson cognitive services that can be accessed in a very similar way that was presented in this post. Using IBM Bluemix services with IBM MobileFirst Platform allow developers to easily and quickly create mobile apps with complex feature, like cognitive question and answer apps, and with great value to the end users.

      The big question now is: -What is the next great cognitive app that YOU will create with Watson cognitive services and IBM MobileFirst Platform ?

      Let us know your thoughts in the comments section and lets keep the conversation flowing

      References:

      IBM Watson:

      Introduction to IBM Watson http://www.ibm.com/smarterplanet/us/en/ibmwatson/index.html

      IBM Watson Developer Cloud http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/

      IBM Watson Question and Answer API Reference http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/doc/qaapi/ and http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/apis/#!/Question_Answer

      Bluemix:

      Access Bluemix now at: https://bluemix.net

      More about Bluemix: http://www.ibm.com/software/bluemix/

      Bluemix developers community https://developer.ibm.com/bluemix/

    Inclusive terminology note: The Mobile First Platform team is making changes to support the IBM® initiative to replace racially biased and other discriminatory language in our code and content with more inclusive language. While IBM values the use of inclusive language, terms that are outside of IBM's direct influence are sometimes required for the sake of maintaining user understanding. As other industry leaders join IBM in embracing the use of inclusive language, IBM will continue to update the documentation to reflect those changes.
    Last modified on May 02, 2016