Skip to content

ADR 0004: Node join is a deployment-layer concern — retire in-product k3s join brokering

Status: Accepted Date: 2026-06-09 Issue: #462

Context

The device-enrollment subsystem hardcodes k3s as the cluster distro. The gateway brokers k3s join credentials (dcs-k3s-token Secret, k3sServerURL flag/ConfigMap, the unauthenticated POST /api/v1/enroll endpoint), the setup wizard's auto-detect reads /var/lib/rancher/k3s/server/node-token from the gateway pod's own filesystem, and dcs-enroll on the device pipes get.k3s.io into a shell to install a k3s agent.

Both reference deployments have been Talos since the demo cutover (#363, 2026-05-12) and the bench rebuild (#408). A k3s agent cannot join a Talos control plane, the node-token path can never exist inside a pod, and no physical controller has been enrolled through this path since the migration. The subsystem is dead-by-construction on every cluster we actually run, yet it still presents live UI (setup wizard auto-detect) and an unauthenticated API endpoint, and the chart still grants the gateway RBAC on the join-token Secret.

Meanwhile the product already has a distro-agnostic adoption path that the Talos world uses today: nodes labeled dcs.io/site=<site> are picked up by the node-discovery reconciler (internal/controller/physical/node_discovery_controller.go), which auto-creates Controller CRs; the Controller reconciler binds a CR to the cluster node of the same name and tracks its readiness. The k3s brokering is a second, redundant, distro-coupled path to the same outcome.

The product-vs-deployment split principle (product ships interfaces + recommended defaults; the deployment instance owns policy, topology, and workflow choices) says a product component hardcoding one distro's join mechanism is the underlying defect — Talos support inside the gateway would be the same defect with a second case.

Decision

Node join is the deployment layer's job; the product retires all in-product cluster-join machinery and keeps only the DCS-level enrollment concerns — Controller CR registration and node binding, dcs.io/-label reconciliation, and audit — behind a documented node-adoption contract.

The contract a freshly-joined node must meet for the DCS to adopt it:

  1. Join — the deployment layer joins the machine to the cluster by whatever mechanism its distro provides (Talos machine config, k3s token, cloud nodegroup, …). The product takes no part in this step.
  2. Registration — one of:
  3. the node carries the dcs.io/site=<site> label at join time, and node discovery auto-creates the Controller CR; or
  4. a Controller CR whose name equals the node name is created in site-<site> ahead of the join (UI "Add Controller" / dcs CLI / GitOps), and the Controller reconciler binds it when the node appears.
  5. Scheduling labels — the physical operator reconciles Controller.spec.nodeLabels (e.g. dcs.io/device=<unit-name>, consumed by UnitSpec.NodeSelector) onto the bound node, so labels no longer need to be applied by join-time flags.
  6. Node requirements — the platform requirements already documented in docs/architecture.md apply: x86_64, hostNetwork port policy (ports outside 32768–60999), container runtime able to pull the unit-runtime image.

Everything that exists only to broker a cluster join is removed from the product: the dcs-enroll binary (and its image/binary release matrix entries), POST /api/v1/enroll and the enrollment-token Secret flow, the dcs-k3s-token Secret and its chart RBAC, the --k3s-server-url flag and its ConfigMap persistence, and the setup wizard's k3s token fields and auto-detect endpoint.

Alternatives Considered

  • Provider abstraction (in-product join behind an interface with k3s and Talos providers) — keeps "enroll a device from the gateway UI" fully self-contained. Not chosen: it grows product surface a solo maintainer must keep working against N distros; Talos join (machine config generation, talosctl secrets) is awkward and privilege-heavy to drive from a gateway pod; and it re-commits the product-vs-deployment violation with more cases.
  • Declare k3s a requirement for device/edge clusters — cheapest, preserves the existing code. Not chosen: it contradicts both of our own reference deployments (demo and bench are Talos), so the vendor would be shipping a path it does not itself run — untestable in practice and misleading to customers.
  • Keep the k3s path as community/experimental tier (per the GPIO precedent) and add Talos docs beside it — not chosen: unlike GPIO, the k3s path is not merely niche but carries an unauthenticated endpoint, secret-brokering RBAC, and shell-interpolation hazards (#471) that would have to be hardened to keep at any tier. Retiring it deletes the hazard instead of maintaining it.

Consequences

  • Surfaces that move: cmd/dcs-enroll/ deleted; scripts/ci-components.tsv and the release/cd workflow matrices drop the component; gateway loses handleEnroll, handleAutoDetectSetup, the k3s setup fields, and --k3s-server-url; the chart drops the gateway dcs-k3s-token Role/RoleBinding (chart-shape change → release-worthy); docs/api-reference.md updates with the route removals (ADR 0003 gate); the physical operator gains spec.nodeLabels → node-label reconciliation.
  • Security posture improves: removes an unauthenticated join-credential endpoint, the gateway's standing read access to a cluster join token, and a curl-pipe-shell + token-in-argv install path on devices (#471's findings die with the code).
  • Sub-issue disposition: #471 closes as superseded (no k3s path retained at any tier); #472 proceeds (setup wizard cleanup); #473 proceeds (docs rewrite against this contract); new sub-issues cover the retirement work and the node-label reconciler.
  • Deployment repos own the join recipe: cndcs-deploy-demo (Operations) documents the golden-path Talos join + labeling; the bench path in #408 already matches the contract.
  • UX trade-off accepted: "enroll a device entirely from two web UIs, no terminal" is no longer a product claim. Joining a node requires deployment-layer tooling (e.g. talosctl). For the pharma target this matches reality: node provisioning is a controlled, change-managed activity, not a self-service flow.
  • Reversibility: moderate. The deleted code is recoverable from history, but re-introducing in-product join would also resurrect the RBAC/endpoint surface and re-litigate this ADR; a future managed-provisioning feature should arrive as a new ADR.