Testing/CI/KubernetesRunners: Difference between revisions

Revision as of 14:58, 17 March 2023

To be able to run Gitlab CI jobs on a Kubernetes cluster, a Gitlab Runner must be installed [1].

Deployment

This sections documents the steps taken to deploy a GitLab Runner instance on a Azure Kubernetes cluster by using Helm [2].

Kubernetes Cluster

Create a Kubernetes cluster on Azure (AKS). Two node pools: "agentpool" for the Kubernetes system pods and "jobs" for the CI jobs.

CLI

Follow the docs to Install the Azure CLI.

Alternatively, run the Azure CLI in a container [3]:

podman run -it mcr.microsoft.com/azure-cli

Install the Kubernetes CLI (kubectl) [4]:

az aks install-cli

Install the Helm CLI [5]:

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Sign in

Sign in to Azure [6]:

az login

Connect to your Kubernetes Cluster. Open the Azure web dashboard for your cluster and push the "Connect" button. A list of commands will be displayed to connect to your cluster. Something like the following:

az account set --subscription ...
az aks get-credentials ...

Gitlab

Register the new runner [7].

Gitlab Runner

Now it's time to install the Gitlab runner with Helm [8].

Create a namespace:

kubectl create namespace "gitlab-runner"

Create a values.yaml file for your runner configuration [9] like the snippet below.

Enabling RBAC support [10] seems to be needed [11] with the default AKS configuration.

gitlabUrl: "https://gitlab.com/"
runnerRegistrationToken: ""
rbac:
  create: true

Deploy the runner:

helm install --namespace gitlab-runner gitlab-runner -f values.yaml gitlab/gitlab-runner

If you change the configuration in values.yaml, apply it with the command below. Pause your runner before upgrading it to avoid service disruptions. [12]

helm upgrade --namespace gitlab-runner gitlab-runner -f values.yaml gitlab/gitlab-runner

@@ Line 5: / Line 5: @@
 === Kubernetes Cluster ===
-Create a Kubernetes cluster on Azure (AKS). Single node pool "agentpool" for the Kubernetes system pods.
+Create a Kubernetes cluster on Azure (AKS).
-Enable virtual nodes [https://learn.microsoft.com/en-us/azure/aks/virtual-nodes] to have on-demand capacity for the CI workloads.
+Two node pools: "agentpool" for the Kubernetes system pods and "jobs" for the CI jobs.
 === CLI ===
@@ Line 34: / Line 34: @@
   az account set --subscription ...
   az aks get-credentials ...
-Register the containers provider, needed for virtual nodes:
- az provider register --namespace Microsoft.ContainerInstance
 === Gitlab ===
@@ Line 55: / Line 51: @@
 seems to be needed [https://docs.gitlab.com/runner/install/kubernetes.html#error-job-failed-system-failure-secrets-is-forbidden]
 with the default AKS configuration.
-<code>nodeSelector</code> and <code>tolerations</code> are needed for the pods to run on the virtual nodes
-[https://learn.microsoft.com/en-us/azure/aks/virtual-nodes-portal#deploy-a-sample-app]
-[https://docs.gitlab.com/runner/executors/kubernetes.html#configuration-example].
 <pre>
@@ Line 65: / Line 57: @@
 rbac:
    create: true
-runners:
-  config: |
-    [[runners]]
-      [runners.kubernetes]
-        [runners.kubernetes.node_selector]
-          "kubernetes.io/role" = "agent"
-          "beta.kubernetes.io/os" = "linux"
-          "type" = "virtual-kubelet"
-        [runners.kubernetes.node_tolerations]
-          "virtual-kubelet.io/provider" = "NoSchedule"
 </pre>