This article is contributed. See the original author and article here.
Last year, Microsoft collaborated with Tigera and released Project Calico to the Windows platform. This contribution to the open-source community was very well received by the community and we got many requests from customers to also support Calico for Windows server on Azure Kubernetes Service (AKS). I’m pleased to announce that Calico for Windows is now available in public preview on AKS.
We have started seeing customers with Linux and Windows operating system (OS) node pools on a single cluster. With more customers adopting containers and microservices, customers are developing applications that runs across Linux and Windows containers. With our support for Calico on Windows in AKS, customer can now have a single network policy experience on node pool cluster running both Linux and Windows containers to secure their network traffic.
In this blog post, I’ll walk you through the steps to install AKS with Calico and setup network policy.
Prerequisites:
- Kubernetes version 1.20.2 or above.
- Bash environment in Azure Cloud Shell (bash)
- Ability to create AKS clusters
- Knowledge on Kubernetes and kubectl
Overview:
- Create AKS cluster with Calico enabled with Direct Server Return (DSR)
- Setup polices for Windows and Linux
- Test the network policy
Step 1: Set up cluster with Calico enabled
We have simplified the process of enabling Calico on a cluster by including it as part of an addon to AKS cluster creation.
Step 1.1: Set the preview feature flag for Calico feature on windows
To use Calico with Windows node pools, you also need to register the “Microsoft.ContainerService/EnableAKSWindowsCalico” flag since it is a public preview feature.
Register the EnableAKSWindowsCalico feature flag by following the command as shown in the following example:
$ az feature register --namespace "Microsoft.ContainerService" --name "EnableAKSWindowsCalico"
You can check on the registration status using running this command below:
$ az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/EnableAKSWindowsCalico')].{Name:name,State:properties.state}"
When ready, refresh the registration of the Microsoft.ContainerService resource provider using the following command:
$ az provider register --namespace Microsoft.ContainerService
Step 1.2: Create AKS cluster with Calico Addons
Here, we will create an AKS cluster with Calico enabled. To enable Calico network policy on Windows, the network plugin must be “azure” since Windows on AKS supports Azure CNI network plug-in only.
Note: When setting up your Windows node pools to your cluster, it is required to add the windows-admin-username and windows-admin-password parameters as you see in the example below.
$ rg=MyResourceGroup
$ location=MyRegion
$ PASSWORD_WIN="MyPassword"
$ az group create -n $rg -l $location
$ az aks create
--resource-group $rg
--name k8s
--node-count 1
--enable-addons monitoring
--windows-admin-username azureuser
--windows-admin-password $PASSWORD_WIN
--kubernetes-version 1.20.2
--vm-set-type VirtualMachineScaleSets
--load-balancer-sku standard
--network-plugin azure
--node-vm-size Standard_D2s_v3
--network-policy calico
$ az aks nodepool add
--resource-group $rg
--cluster-name k8s
--os-type Windows
--name npwin
--node-count 1
--kubernetes-version 1.20.2
--node-vm-size Standard_D2s_v3
When Calico is enabled, you should be able to see the calico system pods running inside the calico-system namespace. To connect to AKS, please refer to connect to cluster doc and run the following command.
$ kubectl get pods -n calico-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-6b4fc665cc-6pmgq 1/1 Running 0 6d
calico-node-474qb 1/1 Running 0 6d
calico-typha-5889b7f49d-9bblq 1/1 Running 0 6d
Step 2: Setup policies on Windows and Linux
Once the cluster is created, we will create client and server pods on Linux and Windows nodes and verify connectivity between the pods.
Step 2.1: Create Linux Pod
Run the following command which includes the yaml file.
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
name: calico-demo
---
apiVersion: v1
kind: Pod
metadata:
labels:
app: busybox
name: busybox
namespace: calico-demo
spec:
containers:
- args:
- /bin/sh
- -c
- sleep 360000
image: busybox:1.28
imagePullPolicy: Always
name: busybox
nodeSelector:
beta.kubernetes.io/os: linux
---
apiVersion: v1
kind: Pod
metadata:
labels:
app: nginx
name: nginx
namespace: calico-demo
spec:
containers:
- name: nginx
image: nginx:1.8
ports:
- containerPort: 80
nodeSelector:
beta.kubernetes.io/os: linux
EOF
Step 2.2: Create Windows Pod
We’ll create a client (powershell) and server (porter) pod on the Windows nodes.
Note: AKS uses Windows Server 2019 as the host OS version only. Container images build using other Windows Server versions are not support. (mcr.microsoft.com/windows/servercore:1809)
- Create powershell pod
Run the following command which includes the yaml file.
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: pwsh
namespace: calico-demo
labels:
app: pwsh
spec:
containers:
- name: pwsh
image: mcr.microsoft.com/windows/servercore:1809
args:
- powershell.exe
- -Command
- "Start-Sleep 360000"
imagePullPolicy: IfNotPresent
nodeSelector:
kubernetes.io/os: windows
EOF
- Create the porter server pod
Run the following command which includes the yaml file.
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: porter
namespace: calico-demo
labels:
app: porter
spec:
containers:
- name: porter
image: calico/porter:1809
ports:
- containerPort: 80
env:
- name: SERVE_PORT_80
value: This is a Calico for Windows on AKS.
imagePullPolicy: IfNotPresent
nodeSelector:
kubernetes.io/os: windows
EOF
You should be able to see all the pods running in calico-demo namespace. Make sure pods are all up and running.
$ kubectl get pod -n calico-demo --watch
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 1 5d19h
nginx 1/1 Running 0 5d19h
porter 1/1 Running 0 5d19h
pwsh 1/1 Running 1 5d19h
Step 2.3: Verify the connectivity between pods
- Verify that the busybox pod can reach the porter pod on port 80.
$ kubectl exec -n calico-demo busybox -- nc -vz $(kubectl get po porter -n calico-demo -o 'jsonpath={.status.podIP}') 80
- If the connection from the busybox pod to the porter pod succeeds, we will get output like the following:
192.168.40.166 (192.168.40.166:80) open
- Verify that the powershell pod can reach the nginx pod.
$ kubectl exec -n calico-demo pwsh -- powershell Invoke-WebRequest -Uri http://$(kubectl get po nginx -n calico-demo -o 'jsonpath={.status.podIP}') -UseBasicParsing -TimeoutSec 5
- If the connection succeeds, we will get output like:
RawContent : HTTP/1.1 200 OK
Connection: keep-alive
Accept-Ranges: bytes
Content-Length: 612
Content-Type: text/html
Date: Mon, 01 Mar 2021 19:00:39 GMT
ETag: "56a78fbf-264"
Last-Modified: Tue, 26 Jan 2016 ...
Forms :
Headers : {[Connection, keep-alive], [Accept-Ranges, bytes],
[Content-Length, 612], [Content-Type, text/html]...}
Images : {}
InputFields : {}
Links : {@{outerHTML=<a href="http://nginx.org/">nginx.org</a>;
tagName=A; href=http://nginx.org/}, @{outerHTML=<a
href="http://nginx.com/">nginx.com</a>; tagName=A;
href=http://nginx.com/}}
ParsedHtml :
RawContentLength : 612
Step 3: Test the Network Policy
Now, we’ll apply a basic network policy to isolate pod traffic.
Step 3.1: Run the Network Policy
In this example, we will only allow busybox to communicate.
$ kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-busybox
namespace: calico-demo
spec:
podSelector:
matchLabels:
app: 'porter'
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app: busybox
EOF
Note: If you want to use calico api, calicoctl is required. It is not available in Azure Cloud Shell by default. You would need to download calicoctl on Azure Cloud Shell. For more information on using calicoctl, please refer to calico docs.
Step 3.2: Validate whether the policy works
- Verify that the busybox pod is still able to reach the porter pod
$ kubectl exec -n calico-demo busybox -- nc -vz $(kubectl get po porter -n calico-demo -o 'jsonpath={.status.podIP}') 80
- Verify that the powershell pod is not able to reach the porter pod
$ kubectl exec -n calico-demo pwsh -- powershell Invoke-WebRequest -Uri http://$(kubectl get po porter -n calico-demo -o 'jsonpath={.status.podIP}') -UseBasicParsing -TimeoutSec 5
The request times out with a message like:
Invoke-WebRequest : The operation has timed out.
At line:1 char:1
+ Invoke-WebRequest -Uri http://10.240.0.122 -UseBasicParsing -TimeoutS ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:Htt
pWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShe
ll.Commands.InvokeWebRequestCommand
command terminated with exit code 1
Conclusion:
Hope you found the following steps helpful in getting started with Calico for Windows on AKS. Give this feature a try and let us know what you think and how we can improve it.
For more details, please see below:
- Network Policy on Azure Kubernetes Services Documentation
- If you have any questions or concerns about AKS, please go to AKS github
- If you like to check details about project calico with Windows
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.
Recent Comments