Customizing Karavel manifests¶

If you have used the Karavel CLI tool to generate a new Karavel GitOps repository, you should find yourself with a huge vendor directory.

This folder contains the Kubernetes manifests for every component managed by Karavel. These manifests have been carefully configured and crafted to work out-of-the-box, but sometimes special configuration must be applied in order to comply with special scenarios or requirements.

Since the vendor directory is managed by the Karavel CLI and changes to its files could be overwritten by future updates, hand-editing them is not recommended. Instead, each folder is provisioned as a Kustomize stack. This allows to edit Karavel manifests by applying a Kustomize overlay that patches the Kubernetes objects.

Patching a vendored resource¶

For example, let's say we want to scale the NGINX Ingress Controller from the default 3 pods to 6, for increased resiliency. The deployment manifests resides in vendor/ingress-nginx/deployment-ingress-nginx-controller.yml, and the field we want to edit is spec.replicas.

Let's start by adding a directory where our patch will live. In the root of your Karavel directory add a new folder called ingress-nginx, at the same level as the vendor directory.

.
|-- ingress-nginx
|-- vendor
`-- # rest of the files and folders omitted

Inside this new directory we'll add our YAML patch. The patch needs to reference the apiVersion, kind, metadata.name and metadata.namespace of the target object, as well as any changes to the rest of the manifest we want to merge. In our case, the new spec.replicas.

# ingress-nginx/deployment-patch.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-nginx-controller
  namespace: ingress-nginx        
spec:
  replicas: 6                     # increase replicas from 3 to 6

Now we need to reference this patch in a new Kustomize stack. Let's add a new file called kustomization.yml to the folder.

# ingress-nginx/kustomization.yml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../vendor/ingress-nginx

patchesStrategicMerge:
  - deployment-patch.yml

As you can see the Kustomize stack references the vendored ingress-nginx directory as a base resource, adding our new file as a patch. This imports all the base Karavel manifests, leaving them untouched except for the controller deployment.

You can check that the patch is being applied correctly by running the following commands:

$ kustomize build vendor/ingress-nginx | grep replicas
replicas: 3

$ kustomize build ingress-nginx | grep replicas
replicas: 6

The directory structure should be something like this:

.
├── ingress-nginx
│   ├── deployment-patch.yml
│   └── kustomization.yml
├── vendor
│   ├── ingress-nginx
│   └── # rest of the Karavel components omitted
└── # rest of the files and folders omitted

Updating the ArgoCD application¶

Adding a patch is only half of the work. Now we need to tell ArgoCD to look it up and apply it when deploying the manifests. By default the ArgoCD applications definitions point to the vendored manifests. We need to change the path to point to our Kustomize overlay.

Here is the Application definition for the NGINX Ingress Controller as generated by the Bootstrap Tool:

# applications/ingress-nginx.yml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: ingress-nginx
  namespace: argocd
  annotations:
    argocd.argoproj.io/manifest-generate-paths: .
spec:
  project: infrastructure
  source:
    repoURL: git@github.com:example/infra.git
    path: vendor/ingress-nginx
  destination:
    server: https://kubernetes.default.svc
    namespace: ingress-nginx
# other fields omitted for brevity

We need to change the spec.source.path field in order to point it to our new folder. We also need to change the annotation for the manifest generation path. This helps ArgoCD detect changes faster by pointing it to the directories this applications is using. Very useful in monorepo situations like a KCP project, where multiple applications are managed with a single Git repository.

# applications/ingress-nginx.yml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: ingress-nginx
  namespace: argocd
  annotations:
-   argocd.argoproj.io/manifest-generate-paths: .
+   argocd.argoproj.io/manifest-generate-paths: .;../vendor/ingress-nginx
spec:
  project: infrastructure
  source:
    repoURL: git@github.com:example/infra.git
-    path: vendor/ingress-nginx
+    path: ingress-nginx
  destination:
    server: https://kubernetes.default.svc
    namespace: ingress-nginx
# other fields omitted for brevity

All is left to do is committing and pushing our changes to the origin repo. ArgoCD will pick up the new Application definition, apply the Kustomize overlay and deploy the changes, including our patched NGINX deployment.

$ git add --all . && git commit -m "Increase NGINX replicas"
$ git push origin master