First look at cdk8s

What is cdk8s?

cdk8s or Cloud Development Kit for Kubernetes is a framework to define Kubernetes resources using a supported programming language of choice which will generate standard Kubernetes yaml manifest files as output.

Basically you define everything in a programming language and get k8s yaml manifests as output.

Why cdk8s?

Plain yaml manifests work fine when you are just getting started with Kubernetes or when you have a fairly small amount of resources to manage. This changes when you add more and more resources to your cluster(s) and need to cover multiple conditions like multiple environments, clusters and other use-cases.

Over time projects like Helm and more recently Kustomize, which try to solve this problem, have been gaining traction. While Helm (the most popular one) uses mainly templating to abstract away complexity, Kustomize has choosen a different approach and uses patching and layering instead.

Both tools hide away most of the complexity for the end user who just wants to deploy an application on Kubernetes. But for people who have a specific use-case and want to make changes to the underlying configuration, it can become a daunting task. Something they probably have to maintain for quite some time too. which in a lot of cases means things will be added and bolted on over time making it painful to debug. Reusability is also often quite hard to do.

So is there anything better? Well, cdk8s might be going into the right direction. Let’s take a look

Example.

In this example we will create a Deployment with a Service and expose it through an Ingress which probably covers a lot of use cases. We’ll make use of the podinfo container.

NOTE: We’ll skip explaining the installation and bootstrapping a new project. The cdk8s docs are clear on how to do this.

By default the main.ts file is the entrypoint from where everything is generated. Let’s go through the file.

You can see that we need to import libraries for the resources we want to create. Next you’ll see that we create a MyChart class which extends the Chart class. The Chart is a container that synthesizes a single Kubernetes manifest.

Inside this MyChart we create a Deployment like shown in the example.

import { Construct } from 'constructs';
import { App, Chart, ChartProps, Size, JsonPatch, ApiObject } from 'cdk8s';
import {Cpu, Deployment, DeploymentStrategy, EnvValue, Ingress, IngressBackend, PercentOrAbsolute} from 'cdk8s-plus-22';

export class MyChart extends Chart {
  constructor(scope: Construct, id: string, props: ChartProps = { }) {
    super(scope, id, props);

    const deployment = new Deployment(this, 'deployment', {
      metadata:{
        annotations: {
          'prometheus.io/scrape': "true",
          'prometheus.io/port': "9797",
        },
        labels: {
          app: 'podinfo',
          team: 'cest',
        }
      },
      strategy: DeploymentStrategy.rollingUpdate({maxUnavailable: PercentOrAbsolute.absolute(0)}),
      containers:[{
        image: 'ghcr.io/stefanprodan/podinfo:6.2.2',
        args: [
          './podinfo',
          '--port=9898',
          '--port-metrics=9797',
          '--grpc-port=9999',
          '--grpc-service-name=podinfo',
          '--level=debug',
          '--random-delay=false',
          '--random-error=false',
        ],
        envVariables: {
          PODINFO_UI_COLOR: EnvValue.fromValue('#34577c')
        },
        resources: {
          cpu: {
            limit: Cpu.millis(800),
            request: Cpu.millis(100),
          },
          memory: {
            limit: Size.mebibytes(500),
            request: Size.mebibytes(100),
          }
        },
        ports: [
          {
            number: 9898,
            name: 'http',
          },
          {
            number: 9797,
            name: 'http-metrics',
          },
          {
            number: 9999,
            name: 'grpc',
          }
        ]
      }]
    })

Now that we have defined our Deployment we need to link it to a Service and expose that using an Ingress. The Ingress controller used in this case is the AWS ALB Controller which needs some specific annotations as you can see in the example.

Apart from the Ingress we also need a backend for the Ingress' host rule. The backend is derived using the exposeViaService method of the deployment object. This is then later added to the Ingress using the addHostRule method.

The TLS configuration is added using the addTls method.

The ingressClassName propery of the Ingress spec is currently not supported by cdk8s but there is a concept of escape hatches that allow you to modify the final manifest during Synthesizing as we did here using JsonPatch.

    const backend = IngressBackend.fromService(deployment.exposeViaService({ports: [{port: 9898}]}));
    
    const ingress = new Ingress(this, 'ingress', {
      metadata: {
        annotations: {
          'alb.ingress.kubernetes.io/scheme': 'internet-facing',
          'alb.ingress.kubernetes.io/target-type': 'ip',
          'alb.ingress.kubernetes.io/subnets': 'subnet-004678903456785,subnet-0b1321321321e212,subnet-06489d92e5839353',
          'alb.ingress.kubernetes.io/tags': 'Application=podinfo, Automation=kubernetes',
          'alb.ingress.kubernetes.io/certificate-arn': 'arn:aws:acm:eu-west-1:09876543211:certificate/56fff548-218d-4f0e-9965-54548697784',
        },
      }
    });
    ingress.addHostRule('k8s-podinfo.mydomain.tld', '/', backend)
    ingress.addTls([{hosts: ['k8s-podinfo.mydomain.tld']}])
   
    // Add ingressClassName (not supported by CDK atm)
    const ing = ApiObject.of(ingress)
    ing?.addJsonPatch(JsonPatch.add('/spec/ingressClassName', 'alb'))

  }
}

The process of converting our code to k8s yaml manifests is called Synthesizing. Each Chart we create will be synthesized into a k8s yaml manifest. You can define multiple Charts and each one will be part of the App construct which can be seen as the root of a tree. You can see this in the last section of our main.ts

const app = new App();
new MyChart(app, 'cdk');
app.synth();

To compile the Typescript code to yaml we need to run the following:

❯ npm run compile && cdk8s synth

> cdk@1.0.0 compile
> tsc --build

Synthesizing application
  - dist/cdk.k8s.yaml

The result is stored in the dist folder and we could run kubectl apply -f dist/cdk.k8s.yaml.

NOTE: you can see the complete project of this example on our GitHub repo (check the dist folder for the full yaml output)

Advantages

Editor / IDE support

Because we can use a programming language we can also benifit from the support or editor/IDE gives us. Think about autopcompletion, linting, syntax highlighting and so on. Oh, and no yaml indentation gotcha’s…

Readability

Long yaml manifests and especially templated onces (like in Helm) can be hard to read. The cdk8s code is more concise and easier to read.

Testing

We haven’t really shown this but the code is actually being tested when we compile the code to yaml. The default test is nothing special but can be extended with whatever you want to test.

Abstractions

Example: Kubernetes API versions

Did you notice that we didn’t need to specify the api versions of the resources we created? cdk8s does that for us. As Kubernetes evolves, API versions get deprecated which means you need to go update all your manifests where this API is used.

To give an example, in Kubernetes v1.22 all beta Ingress APIs (the extensions/v1beta1 and networking.k8s.io/v1beta1 API versions) were deperacted and needed to be updated. This included some changes to how ports were defined and meant quite some work for people that used these. With cdk8s you can re-compile you source using an updated library (cdk8s+)

Overall most of the repeating tidious work is handled by cdk8s.

Constructs

You can create reusable pieces of code called constructs. These constructs enable higher-level abstractions through object oriented classes.

Constructs can also be shared as a package using package managers like npm.

Conclusion

We have just touched the basics here of what cdk8s can do for us but it’s quite clear that it is a step in the right direction. Using a higher level programming language that produces k8s yaml manifests just feels right and opens up a number of interesting possibilities.