Blue/Green Deployment on Kubernetes

6 min readOct 14, 2023

Blue/Green deployment pattern

The concept of blue/green deployment involves the implementation of two identical application deployments, as depicted in the following diagram.

Source: https://cloud.google.com/architecture/application-deployment-and-testing-strategies

Within the diagram, the color blue represents the existing version of the application, while the color green represents the new version. At any given time, only one version remains active and receives traffic. During the creation and testing phase of the green deployment, traffic continues to be directed toward the blue deployment.

Upon the successful completion of testing, the traffic is then redirected to the new version. At this point, you can retain or decommission the blue deployment for potential rollback scenarios. Alternatively, you can utilize the blue environment as a staging area for deploying a newer version of the application.

Overall, blue/green deployment offers a seamless and controlled approach to updating and releasing applications, ensuring smooth transitions and efficient management of different application versions.

Key benefits

Zero downtime
Instant rollback
Environment separation

Considerations

Cost and operational overhead
Backward compatibility
Cutover

Compare with other popular deployment patterns

Blue/Green Deployment on Kubernetes using Helm chart, Skaffold, and Istio VirtualService

To achieve this deployment approach, the crucial factor is identifying a method that allows for the flexible routing of client requests to the Blue or Green environments. In this scenario, I utilize a cookie for routing in Istio, which is established in the Istio VirtualService.

...
      headers:
        cookie:
           regex: ^(.\*?;)?(test=true)(;.\*)?$
...

Source: https://candost.blog/images/content/posts/blue-green-deployment

In my environment, with a GKE cluster utilizing Istio, I intend to employ a Custom Resource Definition (CRD) called VirtualService in Istio. Further details on this strategy will be provided below. If you need to gain a better understanding of Istio, first go through the basic concepts, such as the Gateway and VirtualService. You can see the general folder structure as follows:

go
└──services
   └──example
      └──blue-green-helm-deploy
                   ├── example
                   │   ├── Chart.yaml
                   │   ├── templates
                   │   │   ├── example-deployment.yaml
                   │   │   ├── example-service.yaml
                   │   └── values-prod.yaml
                   ├── manifest
                   │   ├── example-virtual-service-blue-test.yaml
                   │   ├── example-virtual-service-blue.yaml
                   │   ├── example-virtual-service-green-test.yaml
                   │   └── example-virtual-service-green.yaml
                   └── skaffold.yaml

To bring this idea to fruition, I utilize a Helm chart and Skaffold to streamline and expedite the deployment process. The example folder (Helm Chart template) is where the deployment, service, and helm values files are defined.

# example-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.service }}-{{ .Values.release }}-{{ .Values.testColor }}
  namespace: {{ .Values.namespace }}
  labels:
    app: {{ .Values.service}}
    release: {{ .Values.release }}
    color: {{ .Values.testColor }}
spec:
  replicas: {{ .Values.replicaCount}}
  selector:
    matchLabels:
      app: {{ .Values.service }}
      release: {{ .Values.release }}
      color: {{ .Values.testColor }}
  template:
    metadata:
      labels:
        app: {{ .Values.service }}
        release: {{ .Values.release }}
        color: {{ .Values.testColor }}
    spec:
      containers:
        - name: {{ .Values.service }}
          image: {{ .Values.image }}
          command: {{ .Values.cmd | toJson }}
          livenessProbe:
            httpGet:
              path: /health
              port: 8088
            initialDelaySeconds: 10
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /ready
              port: 8088
            initialDelaySeconds: 10
            periodSeconds: 10
          ports:
            - containerPort: 8088
              name: http
          resources:
            requests:
              cpu: {{ .Values.reqCpu | quote }}
              memory: {{ .Values.reqMem | quote }}
            limits:
              cpu: {{ .Values.limCpu | quote }}
              memory: {{ .Values.limMem | quote }}

# example-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: {{ .Values.service }}-{{ .Values.testColor }}
  namespace: {{ .Values.namespace }}
  labels:
    app: {{ .Values.service}}
    release: {{ .Values.release }}
spec:
  type: NodePort
  ports:
    - port: 8088
      name: http
  selector:
    app: {{ .Values.service }}
    color: {{ .Values.testColor }}

# values-prod.yaml
replicaCount: 1
service: example
namespace: test
image: asia.gcr.io/your-project/example
release: prod
reqCpu: 50m
reqMem: 50Mi
limCpu: 100m
limMem: 128Mi
cmd: ['go/services/example/example',
      '--default_env=PROD',
]

In addition, the manifest folder contains the definition files for the VirtualService.

# example-virtual-service-blue-test.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: example
  namespace: test
spec:
  hosts:
  - your-domain.com
  gateways:
  - istio-system/default-gateway
  http:
  - name: example-test
    match:
    - uri:
        prefix: /example
      headers:
        cookie:
          regex: ^(.\*?;)?(test=true)(;.\*)?$
    route:
    - destination:
        host: example-blue
        port:
          number: 8088

  - name: example-default
    match:
    - uri:
        prefix: /example
    route:
    - destination:
        host: example-green
        port:
          number: 8088

# example-virtual-service-blue.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: example
  namespace: test
spec:
  hosts:
  - your-domain.com
  gateways:
  - istio-system/default-gateway
  http:
  - name: example-default
    match:
    - uri:
        prefix: /example
    route:
    - destination:
        host: example-blue
        port:
          number: 8088

# example-virtual-service-green-test.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: example
  namespace: test
spec:
  hosts:
  - your-domain.com
  gateways:
  - istio-system/default-gateway
  http:
  - name: example-test
    match:
    - uri:
        prefix: /example
      headers:
        cookie:
          regex: ^(.\*?;)?(test=true)(;.\*)?$
    route:
    - destination:
        host: example-green
        port:
          number: 8088

  - name: example-default
    match:
    - uri:
        prefix: /example
    route:
    - destination:
        host: example-blue
        port:
          number: 8088

# example-virtual-service-green.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: example
  namespace: test
spec:
  hosts:
  - your-domain.com
  gateways:
  - istio-system/default-gateway
  http:
  - name: example-default
    match:
    - uri:
        prefix: /example
    route:
    - destination:
        host: example-green
        port:
          number: 8088

Lastly, I establish the skaffold.yaml file, which is utilized during the deployment process.

# skaffold.yaml
apiVersion: skaffold/v2beta24
kind: Config
profiles:
  - name: prod-blue
    deploy:
      helm:
        releases:
          - name: example-blue
            chartPath: go/services/example/blue-green-helm-deploy/example
            valuesFiles:
              - go/services/example/blue-green-helm-deploy/example/values-prod.yaml
            setValueTemplates:
              testColor: blue
            artifactOverrides:
              image: asia.gcr.io/your-project/example
            imageStrategy:
              fqn: {}
  - name: prod-green
    deploy:
      helm:
        releases:
          - name: example-green
            chartPath: go/services/example/blue-green-helm-deploy/example
            valuesFiles:
              - go/services/example/blue-green-helm-deploy/example/values-prod.yaml
            setValueTemplates:
              testColor: green
            artifactOverrides:
              image: asia.gcr.io/your-project/example
            imageStrategy:
              fqn: {}

To illustrate the step-by-step deployment of the example service using the blue/green strategy, a diagram will be provided, showcasing the progression of each stage. In the example service, I establish a GET API (/example/get) that confirms the server’s successful deployment by returning its response as “OK”.

# Blue deployment
1.1/ skaffold run -f go/services/example/blue-green-helm-deploy/skaffold.yaml -p prod-blue --tail
1.2/ kubectl apply -f go/services/example/blue-green-helm-deploy/manifest/example-virtual-service-blue-test.yaml
1.3/ curl --cookie 'test=true' --location 'https://your-domain.com/example/get'
1.4/ kubectl apply -f go/services/example/blue-green-helm-deploy/manifest/example-virtual-service-blue.yaml
(1.5/) kubectl delete deployments/example-prod-green -n test

# Green deployment
2.1/ skaffold run -f blue-green-helm-deploy/skaffold.yaml -p prod-green --tail
2.2/ kubectl apply -f blue-green-helm-deploy/manifest/example-virtual-service-green-test.yaml
2.3/ curl --cookie 'test=true' --location 'https://your-domain.com/example/get'
2.4/ kubectl apply -f blue-green-helm-deploy/manifest/example-virtual-service-green.yaml
(2.5/) kubectl delete deployments/example-prod-blue -n test

You can find all the code and files from this demo in my GitHub repository:

GitHub - hungngph/blue-green-deployment

Contribute to hungngph/blue-green-deployment development by creating an account on GitHub.

github.com

Conclusion

In this piece, we jointly explored the application of Blue/Green Deployment on Kubernetes. This approach allows developers to ensure consistent and reliable deployments, minimize downtime, and maintain high availability of applications. It also offers a mechanism to roll back to a previous application version in case of a failed deployment. The combination of Kubernetes, Helm, Skaffold, and Istio VirtualService simplifies the complexities of managing microservices and facilitates continuous development, integration, and deployment.