How To Use Kubernetes Custom Resources To Sell More Burgers

Updated: 03 April 2024

Have you ever heard about the secret menu at McDonald’s?

It’s a set of items that don’t appear on the regular McDonald’s menu, but you can ask for them, and they’ll serve them to you.

Allegedly.

I’ve never been brave enough to ask the server for one of these off-menu items.

And I can’t quite see myself asking for The McGangBang just yet.

But I like the idea that you can take McDonald’s basic primitives – the Big Mac, the McChicken Sandwich, and so on – and remix them to create more interesting products.

This is kind of what Custom Resources do in Kubernetes.

They allow you to extend the menu (the Kubernetes API), and define your own objects, which users can create, manage and delete inside the cluster.

Then, a controller, which watches for these custom objects, can perform actions, like creating a Deployment and a Service to install an app. (Or putting together two McChicken Sandwiches)

Want to make your own custom burger store inside Kubernetes?

In this article we’ll dive into creating a basic Custom Resource Definition!

Let’s go….

Creating a Custom Resource in Kubernetes

(Well that got a bit weird, didn’t it?)

A Custom Resource Definition (CRD) defines the name and structure of a custom object type that you want to allow in your Kubernetes cluster.

A CRD is a way of defining what the object will be called, and what properties it can have: like size, colour or location, for example.

Are Custom Resources a way of storing application data in the cluster?

It might seem like this is a way of storing application data in the cluster, or using it like a database.

Although this is possible, it’s not a reason that I’d recommend that you’d use Custom Resources.

Custom Resources are really about adding your own custom capabilities to your Kubernetes cluster.

Generally, we would use Custom Resources to be able to make something happen in the cluster, like deploy a complex application.

I tend to think of Custom Resources as like instructions to give to a factory, which will then deploy or assemble something in the cluster for you.

For the example, I’m going to keep with the fast food theme. (Can you tell that I’m hungry as I’m writing this?)

We’re going to create a new Custom Resource Definition called the BurgerStore.

As well as running Tutorial Works, I’ve been growing a hugely successful burger franchise, and we’re now expanding into offering our franchisees an ordering app, which will be running on Kubernetes.

We’ll provide the infrastructure (the Kubernetes cluster), and all the franchisees need to do is create a BurgerStore custom resource to get going.

In the case of my CRD, I want it to allow my franchisees to define:

The address of the burger store
Which currency it takes payments in
How many instances of the app should be deployed

Given my requirements, I’ve created this example custom resource definition for BurgerStore:

              apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: burgerstores.tutorialworks.com
spec:
  group: tutorialworks.com
  names:
    kind: BurgerStore
    plural: burgerstores
    listKind: BurgerStoreList
    singular: burgerstore
  scope: Namespaced
  versions:
  - name: v1
    # Each version can be enabled/disabled by Served flag.
    served: true
    # One and only one version must be marked as the storage version.
    storage: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              owner:
                type: string
                description: The name of the owner of this Burger Store franchise
              address:
                type: string
                description: The physical address of the Burger Store
              currency:
                type: string
                description: The currency which this Burger Store accepts payments in
              replicas:
                type: integer
                description: The number of instances of the Burger Store app to deploy
    additionalPrinterColumns:
      - name: Owner
        jsonPath: .spec.owner
        type: string
      - name: Currency
        jsonPath: .spec.currency
        type: string
      - name: Replicas
        jsonPath: .spec.replicas
        type: integer
      - name: Age
        jsonPath: .metadata.creationTimestamp
        type: date

            

You’ll need cluster-admin privileges to be able to do this next command, because creating Custom Resource Definitions is something which happens at cluster-level.

Let’s create the Custom Resource Definition using kubectl:

# kubectl apply -f burgerstore-crd.yml

You’ll get this in response:

              customresourcedefinition.apiextensions.k8s.io/burgerstores.tutorialworks.com created

            

That’s it. Kubernetes has created the new custom resource. The custom resource won’t do anything yet, but this is the first step done.

To check that the Kubernetes API now knows about “burgerstores”, we can use the command kubectl api-resources. This is a handy command that “Prints the supported API resources”.

If we pipe it to grep we’ll see that burgerstores is in the list:

              # kubectl api-resources | grep burger
burgerstores     tutorialworks.com/v1    true    BurgerStore

Great! That’s the first part done.

Now that Kubernetes knows about the concept and structure of a BurgerStore, it will let me create multiple BurgerStore objects.

But, I’m a cluster-admin. I need to allow regular (non-admin) users to be able to create and delete BurgerStore objects. We’ll do that next.

Allowing regular users to create CR instances

Once you’ve created a CRD, it’s generally available in the Kubernetes API, cluster-wide, although you can limit exactly who can create these objects.

I only give my franchisees unprivileged access to the Kubernetes cluster, so I need to allow these regular users to create BurgerStore instances.

If you’re using Kubernetes, you can create a new ClusterRole and grant the permissions to it:

              apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: burgerstores-admin-edit
rules:
  - apiGroups: ["tutorialworks.com"]
    resources: ["burgerstores"]
    verbs: ["get", "list", "watch", "create",
            "update", "patch", "delete", "deletecollection"]

            

And for the view permission:

              apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: burgerstores-view
rules:
  - apiGroups: ["tutorialworks.com"]
    resources: ["burgerstores"]
    verbs: ["get", "list", "watch"]

            

Or if you’re using OpenShift, you can grant permissions to the built-in edit and admin ClusterRoles, by using these special labels:

              apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
items:
  - metadata:
      name: aggregate-burgerstores-admin-edit
      labels:
        rbac.authorization.k8s.io/aggregate-to-admin: "true"
        rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rules:
      - apiGroups: ["tutorialworks.com"]
        resources: ["burgerstores"]
        verbs: ["get", "list", "watch", "create",
                "update", "patch", "delete", "deletecollection"]
  - metadata:
      name: aggregate-burgerstores-view
      labels:
        # Add these permissions to the "view" default role.
        rbac.authorization.k8s.io/aggregate-to-view: "true"
        rbac.authorization.k8s.io/aggregate-to-cluster-reader: "true"
    rules:
      - apiGroups: ["tutorialworks.com"]
        resources: ["burgerstores"]
        verbs: ["get", "list", "watch"]

            

This rather wordy configuration tells OpenShift to give full access to admins (through the admin ClusterRole), and read-only access to limited users who have the view role.

Apply the config above to your cluster to make this happen.

In the next update to this article, we’ll build a website, so I can see and manage all of my burger store franchises, using the Kubernetes API! 🍔