DialogFlow fulfillment —dynamic responses from Google Firestore

Published in

Chatbots Life

14 min readMay 8, 2019

In this little tutorial, I’ll show you how to read information from a database and use it to dynamically generate responses for DialogFlow fulfillment. I have been asked this question by several of my students so I decided to write an article about it. Also, the source code is available here And the DialogFlow export.

This is how the story goes. We are selling drones.

Yes yes, this kind.

And this kind.

And we are doing a bot for Messenger for our clients. Well, we could also use Google Assistant, Telegram any other platform. The point is that for any kind of platform we want to show relevant products. We want to show drones that are available now! To do this we cannot have static responses defined in DialogFlow.

We need dynamic responses. We want to read drones from a database.

The same database that will provide information for a website, for the physical store, and for the mobile app.

This will be a simplified example where we have an intent where the user asks for a specific type of drones. For example a beginner drone or a drone for kids or a racing drone.

Setup the Entity for drone types

To know what kind of a drone the user is looking for, we need to extract the drone type.

We extract values with parameters. And to let DialogFlow know what kind of values/parameters we want to get, we need to define entities.

For extracting parameter values from natural language inputs DialogFlow uses Entities. Entities are like predefined words, objects you’d like to get from the conversation. Therefore we need to define a new entity.

I will create a new custom entity that will define all the possible drone types. To do so click the plus sign and add a name of the entity. I will name it drone-types.

In here we define all the drone types and then synonyms of each one. For example. Drones with a camera could be also called Camera drones. Basically, I would add here all the possible things a user could say. Like this.

I have defined all the possible drone types. And now we can add an intent for buying drones.

Setup Intent for buying a type of drones

I will add a new intent and will call it buy drone. In this intent, we will catch text when a user wants to buy a certain type of drone.

In the training phrases, I will add a few examples of what the user might write. Like: “I need a drone for a beginner” or “I want to buy a drone for my child”.

As you can see DialogFlow already recognizes parameters you want to extract. It knows them because we have defined them as entities.

I will already suggest drone-types be the parameter you can get from the conversation. You could rename this parameter into something else if you wanted.

What I will do is make it required. A column will appear where I can add prompts. That is questions that will be sent to the user when he or she do not specify the drone type.

What I will also do is add an action. We will catch this action later on in the code. From this, we will know that we need to read drones from the database. I’ll add an action buy.drone-types. I just made the name up.

And the last thing we’ll do now is add responses. In the beginning, we’ll just add a few static cards. And we’ll add them for Messenger.

Add static cards in DialogFlow intent for drones

Here we have responses. I will add the Facebook Messenger tab.

We can add different responses for different platforms. Different platforms support different kind of response. Messenger supports these responses:

In this example we will add a simple text and some cards to show different drones on stock. Each drone will have a title, subtitle, image and a link to the page for more information.

And I will first remove the text under default tab:

And in the Messenger tab I will also disable this checkbox. We would have checked this if we wanted Messenger to also get the responses from default tab.

Now I will add a text response. In here I will write “Here is a list of all available $drone-types”. To write out the parameter we use $ and the parameter name.

Now we will add a few cards representing the drones. To each drone, I will add a title, subtitle, image and a button with the links. Like this.

If I now run a query in the test area. Let’s say I want to buy a drone for my kid. I would get a response like this.

But we are not quite satisfied with this. Just imagine in a real shop what happens. One drone is sold out, we get new models, we abandon models that don’t sell.

And now imagine we have to make changes in the website, in the mobile app and the same thing for the chatbot. Too much duplication of work! We want one resource for all the selling points. We want to read the current drones from a database. Let’s enable fulfillment to do so.

Enable fulfillment

To enable fulfillment to go to the intent scroll all the way down, click fulfillment and enable webhook call for this intent. Like this.

There is another checkbox called “Enable webhook calls for slot filling”. If we would enable this one, our app would be called even when the required parameters are not set yet. That would be good if we wanted to validate the parameters.

But what we want is to look for available drones when we know what kind of drones the user is looking for. Therefore lets first figure out how to send cards back to Messenger. Let’s do that. Let’s send the right kind of response.

Return hardcoded cards from the fulfillment

Let’s go to fulfillment. What I will do is use Google Functions to host my fulfillment webhook.

Then you can enable inline editor code and click the Deploy button. It will take a while.

What we could do is use this inline editor to edit the code. I don’t like that. I don’t like to use inline editor. It has limitations. I will download the code after the deployment is done. Download it by clicking this icon in the inline editor:

Now you need to set up a local environment. What you need for that is have Google Cloud SDK installed on your computer. Here is a link to download it. You need to initialize it, select the project you are working on and then come back to this article again. Here is a quick start.

Once you have all set up, open the code you do in your favourite node.js editor. Mine is WebStorm. A popular choice is also Visual Studio Code.

In here we already have some code. Let’s review the modules we use.

The first one is firebase-functions. This one we need to deploy the code to a Google function.

And the second one is dialogflow-fulfillment. This is a class that helps us build the fulfillment code. The great thing about it is that it has functions to work with DialogFlow agent and methods to create responses.

The bad news about it is that it only supports responses for Google Assistant. Therefore if you are creating cards for Google Assistant, you can stick with this one, but if you want to create responses for Messenger like I am trying to in this little article, then you need to create response JSON manually. Yes, we need to create the right kind of JSON response to send back to DialogFlow. Let’s do that.

Let’s first see what is the default json sent from DialogFlow. What is interesting in this json is the property queryResult. It has all the information we are interested in: action, parameters allRequiredParamesPresent, and messages.

allRequiredParamesPresent will always be true since we have enabled webhook only when the parameters are all set. And what we will catch in the fulfillment code is the action. Based on this action we will know we need to read drones from the database. Let’s go back to code.

I will now remove all the unnecessary code and add a few of my own.

I left the firebase functions module and the exports for the function. Function needs to be called dialogFlowFirebaseFulfillment. This was the default name.

In the function I read the action name from request.body.queryResult.action. And then I check if the action is buy.drone-types. In the future agent will have more actions that we will write custom code for.

And now we can write the code for displaying the cards. Let’s first check again how the JSON has to look like.

Here you can see fulfillmentMessages array. This is what we need to build. This kind of JSON. In here we see, that in each message we specify the platform. The first one is a text message. And then two cards are following.

So that are the responses we defined in the DialogFlow interface.

If we now in the fulfillment code read from a database and add another responses, then the responses from fulfillment app will be shown. Therefore double check now that you have fulfillment webhook called enabled. And do not enable slot feeling the webhook call.

And then we can copy the code from the JSON to our app. Just to make sure, we use the right structure in JSON.

What I will do is I will change the first text response, so that I know this response comes from the fulfillment and not from DialogFlow interface.

What you can do now is deploy this project. Go to firebase folder and type

$ firebase deploy

After your code is deployed you can go to DialogFlow test area or into the Facebook Messenger if you have already connected this agent to Facebook and you can try this out.

The response should come from the fulfillment. You will see that the first text is modified.

If you don’t see the new messages, then it means you have some kind of bug in your code. Go to console and write:

$ heroku logs — tail

To see what is happening.

Here is also a link to the code I am using.

Great!

Next step is to create a card dynamically from the information we get from a database.

Read from a Google Firestore database — swap hardcoded drone cards with dynamic cards

This will be a simple demo. In real life, your databases will be more complex and you might use a different kind of a database. In this tutorial, I will use Google Firestore as a database. It is a super simple cloud NoSQL database. It’s flexible, scalable and works fast.

Let’s go to Firebase console and enable it. Click the Create database button.

Under the learn more you can find more documentation if this is your first time using Cloud Firestore.

Click Create database to get started.

We will stay in the locked mode so that only we can use the database. Click Enable to continue.

Now we are ready to add collections to our database. Collections of drones. As said your database that you use in the website and your other resources might look more complex. Here we will simply create collections for each drone type with the drones in it.

Click add collection to start adding collections.

We will make several collections. Each collection will represent one type of drones. Make sure the collection names are the same as the entity name. But remove the white spaces. Here is a list of types we added to DialogFlow.

Your list of collections should look like this:

Now you can start adding documents to each collection. One drone document should consist of title, subtitle, imageUrl and button with a link. What is important is that if you have a different kind of s database structure, this is the information you will have to read from your database. The information that you need to put into the card, that is title, subtitle, URL to the image and a link more.

And since we are creating a simple database our structure can look exactly as the card structure. Therefore like this:

You can fill out the whole database with your products. In my case super cool fancy drones.

Ok! And now how do we connect to the Google Firestore database. Follow me to Project settings:

Under Service account tab you will find a snippet of code for connecting to a database.

You need to generate a new private key, you will use to authenticate to the database. I will save the JSON file in the config folder. Do not share this key on Github or anywhere else. Make sure it is secure.

Once I have it saved I can copy paste the snippet of connection code to my app. And I can change the key path. Like this:

And wow. You are doing super great. Already connected to a database. Now all we need to do is read from it. Nothing easier than that. Follow me!

What I will do first is to read the drone type parameter from the DialogFlow. As we know the request will contain parameters in the queryResult property. Therefore we can access it like this:

And what I will do is remove the empty spaces to get the key we have in a Google Firestore database.

And we are now ready to read from a database. We access the collection by calling collection with the collection name, in our case drone type and then the method get. Like this:

This will return a promise. And after the promise is returned we can loop through each snapshot. A snapshot contains drones in the collection.

We access data in a document by calling the data method. And from this data, we need to structure cards to push them into rich responses. I will first add a text response to the richResponses array. Like this:

I will now add each record into the JSON structure we need to return from this function call.

So, I create a card, that contains the card property and the platform property. I insert each field into the right JSON property. And my fields are named in the exact same way. So it is super simple. Each document I structure and push to richResponses.

And after the loop, I return a richResponses array. I will use them in the next then chain.

In the next then chain i will send the response back to DialogFlow. This response will then be returned to the Messenger bot instead of the one in the DialogFlow interface.