Improve Your Chatbot Using Watson Assistant Chat Logs
A Quick Tour Through the Watson Assistant Improve Panel
Introduction
At the start of your chatbot project, it’s often helpful to identify one goal you’d like to achieve and ways to track your chatbot’s performance against that goal. More importantly, you should also have a plan on how to improve your chatbot until it gets there.
One of the best ways to improve chatbot performance is to monitor user interactions through chat logs. Watson Assistant gives you access to chat logs via the Improve Panel, which is the third section in the Conversation Service Tool. You can access it by clicking in the third icon on the left bar:
The screenshot above shows the Overview section in the panel, which contains some general stats on your chatbot. The component also has a User conversations section that provides a history of all messages from your users, along with the predicted intents and entities for each message. This allows you to observe how your chatbot is performing vis-a-vis its users.
With the Improve Panel, you can:
- Gather all of the messages sent to the chatbot from different channels
- Get basic metrics such as most frequent intents out of all messages or by individual conversations
- Fix messages that have incorrect intents or entities directly in the panel
- Train the chatbot to expand its responses based on new user input
In case you’re wondering how the Improve Panel is different from the Try It Out Panel, the Try It Out Panel is used for you to verify that your chatbot recognizes the correct intents and entities before deployment. The Improve Panel is used to view information about actual conversations that your chatbot has had with your users.
Getting Started With Watson Assistant
Note: This is a quick primer for first-time users of Watson Assistant. If you’re an experienced Watson developer, or going with another platform, feel free to skip this section.
Watson Assistant is a platform that provides developers with the ability to create intelligent chatbots that are able to understand natural language and respond to human intents.
To get started with the service, you need to register for an account. Once you’re registered, create an instance of your Assistant service from the Bluemix dashboard.
To use the instance, launch the Assistant tool and create a workspace.
More on the Improve Panel
Here are the sections inside the Improve Panel:
- Overview: A summary of interactions of users with your chatbot
- User conversations: A list of chat messages. You can update intents and entities while viewing a conversation.
- Recommendations: Ways to improve your system, available only to Premium users.
The rest of this blog post will focus on the User conversations section, which contains a list of chat messages between your users and your chatbot from different messaging channels (Facebook Messenger, Slack, mobile apps etc.).
User Conversation
The User conversation section contains all messages between your chatbot and your users. It lets you easily view all these messages in one place without having to deal with the different messaging channels. These chat logs can also be used proactively to provide early warnings about a chatbot’s misunderstanding of messages.
From the screenshot above, we see that the chat logs are provided in a simple list view. The list can be sorted by Newest or Oldest messages first. This is followed by rows of actual messages, as seen below.
Each row of messages includes:
- The content of the message
- Timestamp in the format of DD/MM/YYY @ HH:MM.
- Predicted intent, and you’re able to edit the intent if it’s incorrect.
- Predicted entities, which you can also edit or make additions to
There is also a search box on top of the messages list. You can search by user statement and/or filter by intents, entities, or dates.
In each row, there is a cool feature called Open conversation.
Clicking that button takes you to the point in the original conversation where the message appeared. This is useful because it brings back the context of the original conversation and helps you understand how the conversation got to that point and where it went afterwards.
Chat Logs API
Finally, Watson Conversation also offers a RESTful API to its chat logs. One useful request you can make is the list of chat logs in a workspace. The request using Curl is shown here:
curl -u “{username}”:”{password}” “https://gateway.watsonplatform.net/conversation/api/v1/workspaces/workspace_id/logs?version=2017-05-26"where version is release date of the version of the API.
This is what a response looks like:
Remember: The API has a rate limit. If the cursor parameter is not specified, the operation is limited to 40 requests per 30 minutes. If the cursor parameter is specified, the limit is 120 requests per minute.
Conclusion
Increasingly, companies are using chatbots for customer support and other operations that simultaneously serve multiple customers. The more users you have, the larger the chat logs.
Being able to effectively examine these chat logs will help you monitor the performance of your chatbot, and thus deliver a better user experience. Hopefully you now have some basic understanding of how to take advantage of Watson Assistant’s Improve Panel. Go forth and dive into those chat logs!
