Simplifying chatbot deployment with Dapr on Kubernetes

by | Jun 1, 2021 | Technology | 0 comments

This article is contributed. See the original author and article here.

This blog post talks about simplifying bot deployments especially in scenarios where the root bot and skill bot architecture is implemented.


 


Prerequisites


 



  • Understanding of Azure Bot Framework.

  • Understanding of Dapr on Kubernetes.

  • A kubernetes cluster such as Minikube, Docker-Desktop with Kubernetes or Azure Kuberenetes Service with ingress configured for SSL/TLS termination. 


 


Introduction


In this blog post we are going to focus on simplifying bot deployments especially in scenarios where the root bot and skill bot architecture is implemented. With this type of architecture, bot functionality is extended by using another bot (a skill). A skill then is a bot that can perform a set of tasks for another bot and hence a bot can be both a skill and a user-facing bot.


 


The below diagram shows how end users interact with root bot and specific skill bots such as a travel booking bot.


 


 

img1


 


The root bot has a public endpoint but if it needs to be in an isolated network, it can be deployed using Direct Line App Service Extension. This is a topic for another blog post.


The skill bot can have public endpoints or private endpoints( not accessible publicly) depending on how the skill bot is consumed. But in the diagram above the skill bots do not have a public endpoint. 


 


Here are a few considerations:



  • The skills bots are typically developed by different teams and deployed at different milestones in the projects. 

  • Some skill bots will have to be live tested(A/B testing) in production to test usage and adoption.

  • Each Skill bots can have different scale requirements. For. e.g Travel Booking bot can have higher scale requirements than an FAQ bot. 

  • From IT point of view, the bots will have to managed from network isolation, authentication, domain names to name a few. 


With the above considerations, we shall see how Dapr helps to fast track some of the development aspects and also aide IT to manage bot deployment in a more stream lined manner. 


 


Dapr – Distributed application runtime enables developers to build a cloud-native application that can be deployed as microservices on-premises, in the cloud or edge devices. Here we are going to treat skill bots as micro services that can be deployed, updated and managed independently just like any other micro service. 


 


Here is what the bot APIs with Dapr can look like on Kubernetes:


img2


 


Firstly, we have Dapr operator deployed on the Kubernetes. The Dapr operator injects Dapr runtime as sidecar containers whenever a bot api container is deployed. Next, we create kubernetes deployment manifest with Dapr annotation as shown below and deploy the bot APIs as containers. 


 


 

  annotations:
        dapr.io/enabled: "true"
        dapr.io/app-id: "simplerootbotapp"
        dapr.io/app-port: "3978"

 


This essentially instructs Dapr sidecar container to communicate with root bot API on its container port 3978.


This is a huge benefit for hosting Bot APIs as it obviates the need for determining unique ports for the Bot APIs and hence the bot ports can be standardized across the board. 


Now, to communicate with the the Root Bot, the Azure Bot Services can connect to the Dapr Endpoint which is:


 


 

https://root-bot-domain-name/v1.0/invoke/simplerootbotapp/method/api/messages

 


 


Notice that we are using TLS between Azure Bot Service and bot api ensuring end-to-end TLS – from web browser to bot api hosted on kubernetes. 


 


In order for the above endpoint to work, the kubernetes needs have the ssl termination configured using ingress controller with the backend serviceport pointing to the dapr port 3500. 


An example ingress configuration is shown below. 


 


 

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: root-bot-ingress
  namespace: root-bot
  annotations:
    kubernetes.io/tls-acme: "true"
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - root-bot-domain.com
    secretName: root-bot-ingress-tls
  rules:
  - host: root-bot-domain.com
    http:
      paths:
      - path: /
        backend:
          serviceName: root-bot-service
          servicePort: 3500

 


 


Now, we can test the root bot with bot emulator and you should get a response as shown below. In the example below, we have a simple root bot that communicates with the end user and simple echo skill bot that echoes user input. 


 


img3


 


Great! Now we are able to communicate with the root bot that is configured with Dapr. Notice in the first log message above, that the bot emulator is connected to the rootbot over https. 


 


Now, if we test a skill, we get the below response. 


img4


 


Yes. We are also able to communicate with the skill bot through the root bot. 


Now, if development teams need to create a new skill bot they would just add a skill manifest configuration as shown below. 


 


 


 

{
  "MicrosoftAppId": "",
  "MicrosoftAppPassword": "",
  "SkillHostEndpoint": "http://localhost:3500/v1.0/invoke/simplerootbotapp/method/api/skills/",
  "BotFrameworkSkills": [
    {
      "Id": "travelbot",
      "AppId": "85be21ea-3bbf-4ae6-a9bc-084718be67fb",
      "SkillEndpoint": "http://localhost:3500/v1.0/invoke/travelbot/method/api/messages"
    },
    {
      "Id": "HRSkillBot",
      "AppId": "85be21ea-3bbf-4ae6-a9bc-084718be67fb",
      "SkillEndpoint": "http://localhost:3500/v1.0/invoke/hrskillbot/method/api/messages"
    },
    {
      "Id": "DepartmentBot",
      "AppId": "85be21ea-3bbf-4ae6-a9bc-084718be67fb",
      "SkillEndpoint": "http://localhost:3500/v1.0/invoke/departmentbot/method/api/messages"
    }
  ]
}

 


 


 


Notice that the SkillHostEndpoint and SkillEndpoint are both pointing to localhost:3500 which is dapr runtime(sidecar container) injected by Dapr operator. 


 


Thus for the development teams, they don’t need to worry about conflicting ports. They just need to provide a unique skill bot api url and dapr runtime would route to the correct skill endpoint. 


For IT, they would expose 443/TLS on kubernetes thus making sure the communication channel is encrypted end-to-end. 


 


In summary, dapr simplifies chatbot deployment comprising of root bot and multiple skill bots and kubernetes provides the scaling capabilities for bot APIs through node pools autoscaling and A/B testing through ingress controllers. 


 


Getting Started


You can test this if you like to see this in action. The artifacts are available in this git repo. This repo has a simple root bot and and an echo bot configured with Dapr. You just to build the containers, publish them to container registry and deploy kubernetes manifest files that describe the bots. 


 


 

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

Submit a Comment