# Compute and Kubernetes Architecture

> VMs, containers and serverless compared, what keeps running when the Kubernetes control plane stops, DRA for GPUs, and the version skew policy.

- Web page: https://www.alesanalytics.com/architecture/compute-and-kubernetes
- Markdown: https://www.alesanalytics.com/architecture/compute-and-kubernetes.md
- Part of: [Enterprise Architecture Atlas](https://www.alesanalytics.com/architecture.md), section "Cloud platforms"
- Last verified: 13 October 2026
- Publisher: Ales Analytics

## Control traded for operating burden

Virtual machines, containers and functions sit on one line. Moving along it from VMs towards functions, you give up control (of the kernel, the scaling, the runtime) and you hand over operating burden in return. None of the three is better. Each fits a different kind of workload and a different kind of team.

| Choose | When | What it costs you |
| --- | --- | --- |
| VMs | Legacy software, host-bound licensing, a few stable applications | Patching and capacity are yours |
| Containers on Kubernetes | Many services, dense packing, GPU sharing through DRA | A cluster control plane, version churn, and a cluster-sized blast radius |
| Serverless | Spiky load and a small operations team | Concurrency limits, cold starts, connection exhaustion, and the highest exit cost |

Serverless deserves a note, because the name misleads. It isn't "no servers". It's managed capacity with a new set of limits: concurrency caps, cold starts, database connections that multiply with every concurrent invocation, and a provider control plane in the path of every scale-out.

The failures to plan for across all three are GPU capacity running out, image pulls failing, and an impaired control plane on the launch path, which is the one that bites during a regional incident.

## Kubernetes in one sentence

Kubernetes is a database of desired state, plus robots that keep making reality match it. The database is etcd. The robots are controllers. Everything else is detail, and most of the detail is about what happens when one of those two stops.

**Figure: What keeps running when the Kubernetes control plane stops.** Control plane: kube-apiserver (every read and write goes through it), etcd (Raft, needs a majority), kube-scheduler (places pods; DRA allocates devices), kube-controller-manager (reconciles until state converges) and admission webhooks in the write path, where an outage with failurePolicy Fail blocks every write, including the fix. On the nodes: the kubelet keeps running pods alive, pods serve with what they have, and ServiceAccount tokens are projected, audience-bound and expiring. When the control plane stops, existing pods keep running; nothing reschedules, scales or receives a new secret. Kubernetes is a database of desired state and a set of controllers that keep making reality match it. When the control plane stops, each kubelet keeps its existing pods running, which is static stability. Nothing reschedules, scales or receives a new secret until it comes back.

### The parts

The API server, etcd (which uses Raft), the scheduler, the controller manager, the kubelet on each node, the CNI for networking, the CSI for storage, CoreDNS, admission control (validating and mutating webhooks), RBAC, ServiceAccount tokens, custom resource definitions and operators, and dynamic resource allocation (DRA) for devices such as GPUs.

### Mechanisms worth understanding

- **Everything goes through the API server, and etcd needs a majority of its members.** Lose quorum and the cluster stops accepting changes.
- **Controllers reconcile, level-triggered.** They compare desired state with actual state and keep retrying until the two converge. They don't react to events so much as to differences.
- **The kubelet keeps existing pods running if the control plane goes away.** That is static stability, and it's real. But nothing reschedules, nothing scales, and no pod receives a new secret until the control plane is back.
- **Admission webhooks sit in the write path.** A webhook that is down, with `failurePolicy: Fail`, blocks every write, including the write that would fix it.
- **DRA allocates devices by attribute.** A pod can ask for a GPU with at least 20 GiB of memory rather than naming a device. DRA's core went to general availability in Kubernetes 1.34 (`resource.k8s.io/v1`). In 1.36, released 22 April 2026, the Prioritized List feature went stable, while partitionable devices, consumable capacity and device taints are beta and enabled by default [1]. DRA allocates devices but doesn't enforce team quotas or fair share, so a queueing layer such as Kueue is still needed.

### Version skew is a constraint, not a suggestion

The upstream skew policy is what turns a Kubernetes upgrade into a sequenced operation:

- API servers in a highly available control plane must be within one minor version of each other [2].
- The kubelet must not be newer than the API server [2], and may be up to three minor versions older.
- The controller manager and scheduler must not be newer than the API server, and may be one older.
- `kubectl` is supported within one minor version either side.
- The API server must not skip minor versions when upgraded.

So "upgrade from 1.33 to 1.36" is three control-plane upgrades in order, with the node pools following behind. Skipping isn't supported.

## Architecture that holds up

A managed control plane. Node pools separated by workload class, with GPU pools of their own. The Gateway API for ingress. Workload identity rather than mounted secrets. Policy enforced at admission. GitOps for desired state. Signed images from a signed registry.

## How a cluster fails

- **etcd loses quorum.**
- **Webhook deadlock:** the admission webhook is down, so nothing can be written, including the webhook's own replacement pods.
- **CoreDNS saturation,** which looks like every service failing at once.
- **API deprecations on upgrade,** where manifests that worked yesterday are rejected today.
- **Cluster-wide bugs in CRDs or operators,** since an operator acts on the whole cluster.
- **An image-registry outage.** Pulls fail only when something scales out or reschedules, so the failure stays hidden until the moment you need capacity.

To these the atlas adds an admission-webhook outage as its own row in the [failure matrix](https://www.alesanalytics.com/architecture/failure.md): every write blocked, detected by API error rate, mitigated with scoped failure policies and a webhook path that runs outside the cluster.

## Trade-offs

**One big cluster or many small ones.** One cluster packs workloads efficiently and makes the cluster the blast radius. Many small clusters contain failures and multiply the platform work. Kubernetes against VMs against serverless is the same trade at a larger scale.

## By provider

EKS, AKS and GKE are E1 for the workload API: the same Deployment runs on all three. They are E2 for everything around it. Workload identity binding differs (EKS Pod Identity or IRSA, AKS Workload Identity, GKE Workload Identity), so do CNI modes, upgrade cadence and support windows. The upstream skew policy is the one thing they share.

> **Kubernetes is not portability.**
>
> The YAML is portable. The platform isn't. Identity, networking, storage classes and upgrade mechanics all differ between managed offerings, so budget for platform work on each provider rather than assuming a lift-and-shift.

## Questions to ask in a review

What keeps running if the API server is gone? Which webhooks fail closed, and where do they run? What's our skew path for the next three upgrades? Is the cluster the blast radius we intend it to be? Where do GPU quotas and fair share live, given that DRA doesn't provide them?

## Verified claims

Each numbered statement above, tied to the page that says it, the date it was last checked and the conditions it holds under.

1. In Kubernetes 1.36 the DRA Prioritized List and admin access are stable, while partitionable devices, consumable capacity and device taints are beta and on by default. Source: https://kubernetes.io/blog/2026/04/22/kubernetes-v1-36-release/. Checked 13 October 2026, tier T1. Holds when: Kubernetes 1.36 upstream. Beta features may be off by default, and a managed service may not expose them yet.
2. Kubernetes API servers in a highly available control plane must be within one minor version of each other, and the kubelet must not be newer than the API server. Source: https://kubernetes.io/releases/version-skew-policy/. Checked 13 October 2026, tier T1. Holds when: Upstream Kubernetes. Managed services (AKS, EKS, GKE) may enforce a narrower window; read the provider's upgrade policy.

Should you have any questions regarding this page, please contact us at solutions@alesanalytics.com.

## Sources

1. [Dynamic Resource Allocation](https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/). Kubernetes, 2026. How Kubernetes allocates devices such as GPUs by attribute, and the feature state of each part.
2. [Kubernetes v1.36 release announcement](https://kubernetes.io/blog/2026/04/22/kubernetes-v1-36-release/). Kubernetes, 2026. Released 22 April 2026. DRA's Prioritized List graduates to stable; partitionable devices and related features remain beta.
3. [Version Skew Policy](https://kubernetes.io/releases/version-skew-policy/). Kubernetes, 2026. The supported version differences between the API server, kubelet, controller manager, scheduler and kubectl, and why minor versions can't be skipped.

## Where to go next

- Previous: [Data, Storage and Databases](https://www.alesanalytics.com/architecture/data-and-storage.md)
- Next: [Security, Secrets and PKI](https://www.alesanalytics.com/architecture/security.md)
- Overview and every page: [Enterprise Architecture Atlas](https://www.alesanalytics.com/architecture.md)
- The whole site in one file: https://www.alesanalytics.com/llms-full.txt