Extension Points in Kubernetes
Use Case: I want to prevent creation of a resource that does not meet standard.
Possible Solution: Validating Webhook
How do you make this? Register a validating webhook with a resource definition that looks like the below example (see the K8s 1.19 API reference). In the webhooks: [] section you can specify any number of webhooks.
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
name: validating-webhook
webhooks: []What are those webhooks? In basic terms, they are a combination of one hook and one or more rules. The rules specify which actions and objects will trigger the hook (example: CREATE a pod). The hook is a service or URL to which kube-apiserver will send an AdmissionReview object (see type definition). Note that this is accomplished via a POST over HTTPS. The hook will then respond with an AdmissionResponse indicating whether to allow admission.
How are hooks implemented? Any technology that can handle a POST over HTTPs can be used such as OPA Gatekeeper using Rego, Spring Boot, or Golang. Note that OPA Gatekeeper has higher-level constructs (via K8s Custom Resource Definitions) that allow specification of common validation rules in declarative form.
How does this help the user? When the user performs an action that involves one of the defined validating webhooks, the webhook is invoked synchronously, meaning that the user’s operation will pass or fail and provide feedback on failure.
Use Case: I want to enrich or modify a new resource with additional information
Possible Solution: Mutating Webhook
How do you make this? Much the same way as the Validating Webhook shown above (see the K8s 1.19 API reference).
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: mutating-webhook
webhooks: []The key difference from a Validating Webhook is that a Mutating Webhook will respond back with a set of JSON Patches that indicate how to mutate the object under consideration. This could include adding labels to a Pod, changing the imagePullPolicy, or setting the container’s privilegeg flag to false .
How does this help the user? The user does not have to navigate the governance policies when pushing objects to different clusters, such as between environments or different security realms. The necessary policies will be applied automatically.
Use Case: I want to create resource X when resource Y is created.
What does this even mean? Let’s take a database example. You want developers using your cluster to have access to a database instance, but you don’t want the developers to have to create and manage the database themselves. So you want “something” that will create the database (resource X) automatically when developers create a database request (resource Y, typically a CRD).
Possible Solutions:
- Validating Webhook
- Mutating Webhook
- Operators
Note that both Validating and Mutating Webhooks can have side effects, but I recommend using the Operator pattern instead (let me know if you would like more information on this). Unlike Validating and Mutating Webhooks, Operators are not Kubernetes objects but rather an abstract pattern. This pattern can have multiple implementation styles. Validating and Mutating Webhooks are both controllers, but not operators (which typically have CRDs). However, the containers and pods in the Operator that implement the control loop can also be called the Controller.
How do you make an Operator? Almost any way you choose! For example, it’s possible to have a Controller that will monitor the K8s API and perform actions. For example, when a user creates a custom resource called MyDatabaseInstance, the Controller will spin up the pods necessary to host a database and then create a ConfigMap in the same namespace as MyDatabaseInstance with the necessary connection details. Just be sure to grant the Controller the necessary permissions (via Role/RoleBinding, ClusterRole/ClusterRoleBinding, and ServiceAccount) to perform the desired operations.
Possible Implementation Stacks:
- Java + Spring + Fabric8io K8s SDK
- Golang (using client-go, which uses the k8s.io/api types also used by K8s itself!)
- Other clients as shown here
- Any stack that can communicate via HTTP(s)
More reading:
