Skip to content

ADR 0013: One blessed golden path for change authoring; the speculative surface around it is demoted, not advertised

Status: Accepted Date: 2026-06-23 Issue: #646 Related: ADR 0012 (same "visible half shipped, load-bearing half absent, tracking issue closed anyway" failure shape), ADR 0005 (product ships interfaces; deployment owns policy). Follow-ups: #647, #648, #649.

Context

Marketing traced the change-control / Promote / review-of-record flow end-to-end while building a buyer-facing figure (cndcs-marketing#51) and could not honestly describe "how you change the plant" without either drawing a confusing matrix or risking a claim that a given authoring×backend combination works when it may not (#646). Engineering did a verification pass against HEAD. What it found:

The model is author → land a reviewable change in a connected dev cluster → e-signed review-of-record at Promote → one of three backends carries the approved change to prod. Around that spine the product has accreted a wide surface: six deployment topologies (A–F), three authoring on-ramps, and three backends. Verifying each cell of that matrix showed the spine is sound but the surface is uneven — parts are wired and tested, parts are assembled-but-unproven, and parts are contracts with no demonstrated consumer.

Authoring on-ramps:

  • Browser edit → Save → Promote is wired and is the real path. Save is a direct dev-cluster write (no ChangeRequest); the reviewable, e-signed ChangeRequest is created at Promote.
  • dcs apply to a dev cluster → Promote is wired (dcs apply POSTs to the gateway; a ChangeRequest manifest creates a signed CR).
  • Offline dcs edit → git push → a GitOps-watched dev cluster → Promote is an assumed bridge: the product deliberately stays Git-agnostic and writes YAML to disk; the "push lands in a dev cluster" half is a deployment concern (Flux/kubectl) with no product wiring and no test.

Backends:

  • direct-apply — implemented, unit-tested (SSA).
  • git-pr — GitHub implemented and unit-tested against a fake API; Gitea is a hard stub (defaultProviderFactory → ErrUnsupportedProvider). The full chain (PR → merge → Flux → audit-flux-bridge → AuditRecord with gitCommitSHA) is never run as one end-to-end test; the halves are tested separately plus one integration test that pre-stamps the CR (#648).
  • in-cluster-queue — a ~5-line stub returning Pending forever, waiting for a deployer-supplied external system to patch status.phase. No reference integration, no consumer, no e2e proving the external-release half works (#649).

The review-of-record itself is narrower than it looks. fetchCurrentTarget (internal/gateway/changerequest.go:267) feeds current cluster state to the Promote diff for MasterRecipe only; every other engineering kind falls back to "no current state" → raw-YAML diff. #292 closed claiming all 19 renderers "feed the Promote UI," but the renderers are client-side and are never fed current state for 18 of 19 kinds — the server-side fetch was never generalized (#647). So "review any change at Promote" is today "review recipe changes semantically; everything else is raw YAML."

The tension: the spine is good and worth describing to buyers as one flow, but the surrounding optionality is partly speculative and, described as a matrix, makes the product look both more capable and less trustworthy than it is. We need a decision about what is blessed, what is demoted, and what must be true for the 21 CFR Part 11 review-of-record claim to hold.

Decision

There is one blessed golden path for authoring a change, and everything else in the change-control surface is positioned relative to it — demoted to a documented convenience or marked experimental — rather than advertised as a co-equal option.

The golden path is:

Topology B (one cluster, dcs-dev / dcs-prod namespaces) → browser edit → Save to dev → run a sim batch → Promote (semantic diff + electronic signature) → distinct approver signs → direct-apply carries it to prod (or git-pr via GitHub where Git is the system of record).

Concretely:

  1. Bless Topology B as the golden path; Topology D (laptop-mode against a local kind/k3d cluster) is the supported power-user variant. Docs, marketing, and examples describe this one flow. B exercises every product interface and scales to C (two clusters) without changing the model.

  2. Demote Topology F (filesystem authoring) from a co-equal topology to a documented dcs edit / dcs diff CLI convenience. dcs edit/dcs diff stay shipped and useful for a cluster-free authoring window and for previewing a change in one's own Git review. But the F → dev-cluster → Promote bridge is not advertised as a supported, verified path unless and until it is wired and tested: since review and prod both require a cluster, and a local kind/k3d cluster (Topology D) already gives a near-zero-cost connected authoring loop, the offline window F buys is marginal and does not justify presenting F as a first-class topology.

  3. git-pr is GitHub-only. Docs state this plainly; Gitea remains a stub with the Provider interface seam exposed for deployers who wire their own.

  4. in-cluster-queue is experimental. It is documented as "experimental — no reference integration" and removed from the co-equal-backends framing until a design partner actually runs a ticket-driven SOP that demands it (#649).

  5. The 21 CFR Part 11 review-of-record claim is qualified until the current- state fetch is generalized. "Review any change at Promote" is only honest once fetchCurrentTarget feeds current state for all engineering kinds (#647); until then docs say the semantic review-of-record covers recipes and the raw-YAML review-of-record covers everything else.

This decision changes positioning and honesty, plus three scoped code/test follow-ups (#647, #648, #649). It removes nothing that works.

Alternatives Considered

  • Wire and test the offline F → dev-cluster → Promote bridge as a first-class path (product gains a dcs push/reconcile-into-dev step, with an e2e test). Rejected as the target: it adds Git-credential/remote/merge surface the product deliberately avoids (gitops-for-automation-engineers.md), to buy an offline window that Topology D already covers with a local cluster. Kept on record as the move if a design partner needs true air-gapped authoring.

  • Keep all six topologies and three backends as co-equal, just add the missing tests and the generalized fetch. Rejected: even fully tested, presenting a 6×3 matrix to a buyer is the exact problem #646 raised — it reads as either bewildering or untrustworthy. The fix is a blessed default and honest framing, not just more green checkmarks. (The tests in #647/#648 are still worth doing; they are necessary, not sufficient.)

  • Delete the speculative surface outright (in-cluster-queue backend, the dcs edit filestore path). Rejected: dcs edit/dcs diff have real convenience value and shipped cleanly (#450/#451); the in-cluster-queue seam is 5 lines and a genuine integration point for regulated customers with existing CM systems. Demote and mark experimental, don't amputate.

  • Bless Topology D (laptop-mode) as the golden path instead of B. Rejected: D is per-engineer and assumes comfort with the model; B is the team default that exercises pod-mode, admission policy, and Flux reconciliation in one cluster. D is the power-user variant of the same flow, not a different one.

Consequences

  • Docs that move:
  • docs/explanation/ui-deployment-topologies.md: F is recharacterized as a dcs edit/dcs diff CLI convenience, not a co-equal topology; the F→dev→Promote bridge is described as a deployment responsibility, not a verified product path. B is stated as the golden path, D as its power-user variant.
  • docs/change-control.md and docs/platform-administration/changerequest-backends.md: git-pr is GitHub-only (Gitea stub); in-cluster-queue is experimental / no reference integration; the review-of-record is semantic for recipes and raw-YAML for other kinds until #647 lands.
  • docs/batch-operations/promote-recipe.md: the misleading "renderer hasn't been written yet" empty-diff message is corrected as part of #647.

  • A single "golden path" figure/section that marketing (cndcs-marketing#51) can redraw honestly: one flow, not a matrix.

  • Code/tests (the follow-ups this ADR blesses):

  • #647 — generalize fetchCurrentTarget to a dynamic-client GET so every engineering kind is fed current state at Promote; this is what makes the "review any change" claim true. Highest value.
  • #648 — end-to-end test for the git-pr full chain, or an explicit "assembled, not e2e-verified" doc note.

  • #649 — mark in-cluster-queue experimental (or ship a reference consumer

    • test).

  • Compliance: docs/compliance/21cfr11.md should record that the §11.10(b) "accurate copies / review of records" posture for engineering changes is, at the time of this ADR, semantic for recipes and raw-YAML for other kinds, closing to fully-semantic with #647. The electronic-signature and segregation-of-duties guarantees are unaffected — they hold across all kinds and backends already.

  • Marketing: unblocks the §01 "the plant lives in Git" figure — one blessed flow to draw, with the honest current-state caveat on the diff until #647.

  • What does NOT change: the ChangeRequest lifecycle, e-signature/HMAC flow, segregation of duties, admission gating, direct-apply, and git-pr-via-GitHub all stand. dcs edit/dcs diff keep shipping. No CRD or API contract changes.

  • Reversibility: high. This is primarily a positioning/docs decision plus three scoped follow-ups; re-promoting Topology F or in-cluster-queue to first-class later is a docs change once the wiring/tests that would justify it exist.

  • Process note (why this ADR exists at all): the fetchCurrentTarget gap is the second instance of the ADR-0012 pattern — a tracking issue (#292) closed on the strength of the visible half (renderers) while the load-bearing half (the fetch that feeds them) was never built, and no test asserted the end-to-end capability. The standing lesson: a "feeds the UI / executes / is reviewable" claim is not done until a test exercises it end-to-end, not just the half you can see.