Developing Answer Bot for Customer Service
About 4 years back, for some situation, I had filed few customer queries about a product operating procedures. As auto generated response, I received highly canned and irrelevant response. Having worked in NLP area for sometime, I started an experimental project to fix it. Within 2 months, I developed and launched Botreload Agent Assist Bot at Zendesk Marketplace. At its peak, it was serving 50+ clients with 100K queries per day.
Struck by Giants:
Marketplace leaders are always in position to mine the data to get to most flourishing land in their marketplace and capture it by offering an integrated product offering.
By end of 2018, Zendesk launched their integrated offering with their own Answer Bot. That was death knell for my Bot experimental project. Nevertheless, it was great learning experience to single-handedly launch an App into Enterprise Marketplace. Yey! :)
Open Sourced:
Beauty of this solution is its fully automated pipelines to Curate and Generate new Intents and Responses regularly for each business unit and onboarding-serving-retraining process without any Ops requirement. I am sure, many may be curious to know its How. Although, it does not use DNN (Deep Neural Network), it is able to perform great. To share knowledge, I have decided to Open Sourced it at my Github.
Most Popular Chatbot Tutorials:
1. Build a Telegram Bot Scheduler with Python
2. A Conversational UI Maturity Model: a guide to take your bot to the next level
Here in this article, sharing its design details with open source community.
About Botreload Agent Assist:
Botreload Agent Assist’s Smart Reply system automatically generates and reply to incoming Ticket query using AI Tech (NLP/NLG/ML/Deep Learning).
Key Features:
- Suggest most Relevant Reply — Suggest quick and most relevant reply to customer query. Its algorithm is design based on Research paper published by Google.
- Cold Start Capability — Engine is trained on large enterprise helpdesk data to start serving without even any custom training.
- Continuously Serve and Learn — Its AI Engine learns from everything from past as well as present in near-realtime. It can even serve without any training (Cold start).
- Automatic ticket tagging — Agent Assist understands the content of each ticket and categorize it accordingly using both existing tags as well as newly discovered ones.
- Generates Performance Analytical Dashboard by business unit
Key Technical Features:
- Automatically curates and generates new Query’s Intents for each business unit separately
- Automatically curates and generates new Smart Replies for each business unit separately
- Fully automated process of onboarding to serving Pipelines.
Architecture:
This bot has three major parts — Agent Assist App, Smart Reply API and Analytics Components .
Agent Assist App:
- Agent Assist App, deployed in Zendesk Marketplace, reads customer queries and suggest Smart Reply with confidence level.
- It presents a minimal user interface to suggest smart reply for each query by interacting with Zendesk APIs and Smart Reply APIs.
Smart Reply API:
- Smart Reply AI Engine, deployed on Google Cloud Platform, serves Smart Reply through APIs for incoming query from Client App.
- It has three major components Data Extraction Bot, Smart Intent Predictor Bot and Smart Reply Predictor.
- It has three major Pipelines to serve Smart Reply capability. (Discussed below)
Analytics Components:
- These are AI Model Monitoring and Performance Dashboard components.
AI Pipelines:
Beauty of this solution was its Intent and Smart Reply Prediction Pipelines which works without any training data for a particular business unit. In following sections, I reveal these in details.
Smart Intent Predictor Pipeline:
This pipeline periodically curates new Intents by clustering queries from historical data then build Intent Predictor model using these new Intents. This is done within a business unit data to maintain data confidentiality.
Following are steps for Smart Intent Predictor Pipeline.
- Create query clusters using Query Historical data to find most important query Topics
- Generate Intent and its central queries per cluster to find out new Intents and keywords/phrases to train the model.
- Train Cluster Intent Model with Intents with their Keywords/phrases to predict Intent for their clusters.
- Label Historical Query data using Cluster Intent Model to generate training data for final Smart Reply Intent model.
- Train Smart Reply Intent model using new labeled Query data to predict customer query intent.
Smart Reply Predictor Pipeline:
This pipeline periodically generates new Responses by summarizing over a cluster of response. Then these responses are fed to train Smart Reply model to qualify for incoming query intents. Again, this is done within a business unit data to maintain data confidentiality.
Following are steps Smart Reply Predictor Pipeline.
- Create query clusters using Query Historical data to find most important query Topics
- Generate Response cluster for each query cluster using Query Cluster and Historical Response Data
- Generate new Responses by summarizing over Response clusters and map Query Clusters with it using Smart Intent Predictor model.
- Train Smart Reply Predictor model using mapped Intents and their Response Summaries
Smart Reply — Serving Pipeline:
This pipeline is a Serving Model Pipeline to predict Smart Reply using above two Predictor models. These models are saved in GCP Storage, separated by each business unit. When API request arrives, these are loaded in memory just-in-time to respond to request.
Following are steps of serving Smart Reply model online.
- Feed Customer query into Smart Intent Predictor to get its Intent
- Predict Response from Smart Reply Predictor using Intent
Conclusion:
I work full-time as Product Engineering Leader where I have been leading multiple teams to develop enterprise products. But developing a fully automated AI Bot system single-handedly was a worthwhile journey. It has certainly sharpen my Strategic and Execution abilities to see both opportunities and pitfalls beforehand.
Hope, open source community folks find this sharing useful. Please do comment/question, will be glad to interact.
Github: https://github.com/saurabhkaushik/botreload-agent-assist
Website: http://botreload.com/product_agentassist.html
Don’t forget to give us your 👏 !
Saurabh Kaushik: Data and AI Product Management Leader for 24 years. From web 1.0 to cutting-edge AI solutions, he’s pioneered tech products across industries, from startups to enterprises. Saurabh is a renowned thought leader and speaker at global tech forums, and his tech blogs span over a decade. His relentless innovation continues to shape Data and AI solutions worldwide.
Connect with him on Linkedin
