Host on Kubernetes

If you already run a Kubernetes cluster, the site fits on it as a small deployment. The repository includes the manifests and a Containerfile that packs the whole site into one small image. If you only need one server and already have a web server on it, plain files are simpler.

This guide is about hosting a copy of this website. The cluster the project’s own pipeline is designed to run on is a different thing, described in Cluster architecture.

What you need#

• A Kubernetes cluster and kubectl set up to talk to it.
• A container registry your cluster can pull from.
• Podman, to build the image. It needs no background service and no root access. Any tool that builds a standard image from a Containerfile works the same way, including Buildah, nerdctl and BuildKit.

There is no published image to pull. You build your own, so you know exactly what is in it.

What gets created#

The manifest in k8s/clearedstack.yaml creates three things:

1. IngressIngress controllerRoutes clearedstack.example.com and terminates TLS.
2. ServiceClusterIP servicePort 80 balances requests across pod replicas in the namespace.
3. PodsTwo deployment replicasPod replicas serve port 8080 with probes and resource limits.
4. Namespaceclearedstack namespaceIsolates the deployment, service and optional network policy.

There is no database, no persistent volume, no config map and no secret. The pods hold nothing that needs backing up.

Build and push the image#

Replace registry.example.com with your registry’s address.

git clone https://github.com/ClearedStack/clearedstack.com.git
cd clearedstack.com
podman build -t registry.example.com/clearedstack:1 .
podman push registry.example.com/clearedstack:1

The build runs in two stages. A Node.js container installs the dependencies and builds the site into plain files. Those files are then copied into a small nginx image, and the Node.js stage is thrown away. What is left holds a web server and the finished pages: no database, no application server and no source code. The web server runs as an ordinary user, not as root.

Use a new tag for every build, such as :1, :2 or the git commit. A tag that never changes its meaning is what lets you roll back.

Point the manifest at your image#

Open k8s/clearedstack.yaml and change the image: line to the name you just pushed:

image: registry.example.com/clearedstack:1

If your registry needs a login, create a pull secret in the namespace and name it in the pod spec, next to containers::

imagePullSecrets:
  - name: registry-login

Deploy#

kubectl apply -f k8s/clearedstack.yaml

1. BuildBuild container imagepodman build reads the Containerfile and produces an unprivileged nginx image with the static files.
2. PushPush to registryThe image goes to your own registry under a tag you never reuse, such as :1.
3. ConfigureUpdate manifestSet image line in k8s/clearedstack.yaml to match pushed registry tag.
4. ApplyApply manifestkubectl apply creates the namespace, deployment and service.

Wait for it to finish starting:

kubectl -n clearedstack rollout status deployment/clearedstack

Then look at what is running:

kubectl -n clearedstack get pods,service

You should see two pods with a status of Running and 1/1 ready.

Check that it works#

Forward a local port to the service:

kubectl -n clearedstack port-forward service/clearedstack 8080:80

Open http://localhost:8080. Press Ctrl-C to stop forwarding.

Put it on your own domain#

How you expose a service depends on your cluster. If it has an ingress controller, the repository has an example to start from:

DecisionHow does your cluster route incoming traffic?

• Ingress controller
  k8s/ingress.yamlUse an Ingress resource with TLS secret and route to port 80.
• Gateway API
  HTTPRoute resourceRoute your Gateway listener directly to the clearedstack ClusterIP service.
• Load balancer
  LoadBalancer serviceExpose service type LoadBalancer on port 80 directly from your cloud provider.

cp k8s/ingress.example.yaml k8s/ingress.yaml

Edit k8s/ingress.yaml to set your host name and the ingress class your cluster uses. For HTTPS, add a tls section that names a secret holding your certificate:

spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - clearedstack.example.com
      secretName: clearedstack-tls
  rules:
    - host: clearedstack.example.com

Then apply it:

kubectl apply -f k8s/ingress.yaml

If your cluster uses the Gateway API or a load balancer service instead of ingress, point it at the service named clearedstack on port 80.

If your copy lives at a different address than clearedstack.com, change url in src/data/site.ts before you build the image.

What the manifest locks down#

The pods are set up to need as little trust as possible:

• They run as an ordinary user, not root.
• The container’s filesystem is read-only, apart from a small scratch folder.
• All Linux capabilities are dropped and privilege escalation is off.
• The default seccomp profile is applied.
• They get no Kubernetes API credentials.
• Each one asks for 16 MB of memory and is capped at 64 MB.

These settings are written to meet the “restricted” Pod Security Standard. To have the cluster enforce that standard on the namespace, so nothing less strict can ever run there, add a label:

kubectl label namespace clearedstack pod-security.kubernetes.io/enforce=restricted

Block outbound traffic#

The site never needs to open a connection to anything. If your cluster’s network plugin supports network policies, you can make that a rule. Save this as k8s/networkpolicy.yaml and apply it:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: clearedstack
  namespace: clearedstack
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: clearedstack
  policyTypes: ["Ingress", "Egress"]
  ingress:
    - ports:
        - port: 8080
  egress: []

The pods can then receive requests on port 8080 and send nothing. If you want to accept traffic only from your ingress controller, add a from rule that selects its namespace.

Keep it up during maintenance#

Two pods mean one can be replaced while the other serves. To stop a node drain from taking both down at once, add a pod disruption budget:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: clearedstack
  namespace: clearedstack
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: clearedstack

To change the number of copies:

kubectl -n clearedstack scale deployment/clearedstack --replicas=3

One pod is enough for a small team. The site is plain files and each request is cheap.

Update it#

Build and push the new version under a new tag, then tell the deployment to use it:

git pull
podman build -t registry.example.com/clearedstack:2 .
podman push registry.example.com/clearedstack:2
kubectl -n clearedstack set image deployment/clearedstack web=registry.example.com/clearedstack:2

Kubernetes replaces the pods one at a time, so the site stays up during the change. If the new version is wrong, go back:

kubectl -n clearedstack rollout undo deployment/clearedstack

If you keep your manifests in git, change the image: line there and apply the file instead of using set image, so the file and the cluster stay in step.

Move it to a closed network#

Build the image on a connected machine and save it to a file:

podman save registry.example.com/clearedstack:1 | gzip > clearedstack.tar.gz

Carry the file across by whatever transfer process your network requires. On the other side, load it, give it the name of the registry inside that network, and push it there:

podman load -i clearedstack.tar.gz
podman tag registry.example.com/clearedstack:1 registry.internal.example/clearedstack:1
podman push registry.internal.example/clearedstack:1

Then set that name in k8s/clearedstack.yaml and deploy as above. Run it on a closed network covers checksums and verification.

When something goes wrong#

Start with these two commands. The events at the end of the first one usually name the problem.

kubectl -n clearedstack describe pod -l app.kubernetes.io/name=clearedstack
kubectl -n clearedstack logs -l app.kubernetes.io/name=clearedstack

Remove it#

kubectl delete namespace clearedstack

This removes the deployment, the service, the ingress and any policies in one step.