I recently had the opportunity and need to check out CDK8S and how my team might be able to use this tool to programmatically generate Kubernetes manifests for a near infinite number of services we are building for our move to Kubernetes.

As you will begin to read, there are inception levels of abstraction going on around this library, which can make troubleshooting and building around this project very confusing, but there is still a lot of power and good things going on that make some of the pain worth it.

What is CDK8S?

CDK8S is an alpha level library that allows you to write high level abstractions of Kubernetes objects like deployments, services, and more all in your favorite language ( TypeScript, Python, and others).

It is very new, and nearly 99% of the existing documentation is developed for TypeScript. That is because the core of CDK8S is all TypeScript, and if you use another language that has to be synthesized to typescript with the AWS super library of jsii to translate other languages into TypeScript. So if you are following we go from Python > TypeScript > Kubernetes Manifests > kubectl apply.

The big selling points are that if your team prefers using one language above others and to keep logic and tooling all in the same ecosystem of known stuff than cdk8s makes a great use case. It helps keep code DRY and reusable and potentially reduces cognitive load for the team.

A Walkthrough

Now that we have a somewhat formal concept of what CDK8s is trying to do and why one would use it, let’s try it out.

Install the CLI

The CLI tool is entirely written in TypeScript, So one must use npm to install. The CLI tool is NOT written in Python. Of course due to this fact you must have npm installed on your system.

$ npm i -g cdk8s-cli

Start a new project

$ mkdir cdk8s-project
$ cd cdk8s-project
$ cdk8s init python-app

This will setup a pipenv with the most recent versions of the python cdk8s libraries and create a main.py file which is what we will be using.

Working with Python CDK8s

At this point you have everything setup and you’ve read the examples but really what you want to know is how to do the thing you would normally type for Helm or direct manifests to create specific options for a deployment or service in Kubernetes.

Where is the documentation? You ask yourself. You google and come up basically blank. Where is the PYTHON documentation for the cdk8s library? Can’t find it in github, a little bit more in pypi but not enough to formalize what needs done to make something like resource limits for a Pod. Then you re-read the example file.

5 from imports import k8s

It takes a leap but then you notice a directory in cdk8s-project that you hadn’t before. A little navigational deep dive reveals the location of a single __init__.py file with 48000+ lines. Where did this file come from? No one knows.

Actually what is happening, is that jsii is synthesizing the TypeScript library back into Python so that we can use it.

This file is the only real documentation for the Python CDK8S library. As stated previously, it is enormously long, which makes it a bit difficult to fully grok in a text editor.

A trick I learned just this week is that one can use pydoc to render the Doc Strings as a webpage, where Ctrl F works.

Open a new terminal shell

$ cdk8s-project
$ pipenv shell
$ pydoc -b
# A webpage will automagically open

Now you finally have documentation that will allow you to build the things you want beyond the most basic examples.

#!/usr/bin/env python
from constructs import Construct
from cdk8s import App, Chart

from imports import k8s # use pydoc to read this lib in browser

class MyChart(Chart):
    def __init__(self, scope: Construct, id: str, replicas: int, application_name: str):
        super().__init__(scope, id, 
            labels = {"project": application_name, "service": id}, 
            namespace="application"
        )

        KCM = k8s.KubeConfigMap(self, "configmap",
            metadata=k8s.ObjectMeta(name=id),
            data={
                'ENVIRONMENT': 'dev',
                'AWS_DEFAULT_REGION': 'us-east-1'
            }
        )
        

        k8s.KubeDeployment(self, 'deployment',
            metadata=k8s.ObjectMeta(name=id),
            spec=k8s.DeploymentSpec(
                replicas=replicas,
                selector=k8s.LabelSelector(match_labels=self.labels),
                template=k8s.PodTemplateSpec(
                metadata=k8s.ObjectMeta(name=id, labels=self.labels),
                spec=k8s.PodSpec(containers=[
                    k8s.Container(
                    name=id,
                    image='private-repo.io/container:latest',
                    ports=[k8s.ContainerPort(container_port=443)],
                    env_from=[k8s.EnvFromSource(config_map_ref={
                        'name': KCM.name
                    }
                    )],
                    resources=k8s.ResourceRequirements(
                        limits={"cpu":k8s.Quantity.from_string("4096m"), 
                                "memory":k8s.Quantity.from_string("8192Mi")
                                },
                        requests={"cpu":k8s.Quantity.from_string("1024m"), 
                                "memory":k8s.Quantity.from_string("2048Mi")
                                }
                    )
                    )]
                )
                )
            )
        )

        k8s.KubeService(self, 'service', 
            metadata=k8s.ObjectMeta(name=id),
            spec=k8s.ServiceSpec(
                type='LoadBalancer',
                ports=[k8s.ServicePort(port=443, target_port=k8s.IntOrString.from_number(443))],
                selector=self.labels
                )
            )



app = App()
project = "superduper"
MyChart(app, "www", 2, project)
MyChart(app, "api", 3, project)

app.synth()


# from cdk8s-project/
$ cdk8s synth

This will make 2 files under dist/ called www.yaml and api.yaml, each with their own deployment, service, and ConfigMap.

At this point you can use kubectl to apply the manifests from dist/.

And now you feel the power and awesomeness of this tool. You can go from 1 -> n services. You can do a lot of templating and adjustment based on projects very quickly.

CDK8S Conclusion

The wish to reduce cognitive load for building these kubernetes manifests with a language like Python is never quite reached, but I believe that with a few more revisions and the nicer library of cdk8s+(Python documentation for cdk8s+ appears to be in even more of a hard to find place) will come together to make this a very nice tool to use. It’s just not fully there yet. If you can stomach a bit more of the DIY nature and figuring out how to navigate some of the difficulties with the documentation and seeing typescript errors within python stack traces you will succeed. And maybe if your team uses TypeScript a lot of these challenges go away, but I find it highly confusing trying to track the difference between all the docs written for TypeScript and make that work in Python.