Audit Linkage (Git commit → AuditRecord)¶
This guide is for the platform administrator who has already turned on
GitOps enforcement and wants the resulting
AuditRecord objects to carry the originating Git commit SHA, closing
the §11.10(k) traceability loop introduced in
#283.
Flux only, today
Audit linkage is implemented for Flux as the reconcile engine. The
bridge parses Flux notification-controller events specifically.
ArgoCD, ServiceNow, and other engines currently have no path to a
SHA-stamped AuditRecord, tracked in
#816.
Admission enforcement itself is engine-neutral
and works with any of them. Only this linkage half is Flux-coupled. To
prevent a silent gap, the chart refuses to render when enforcement is
enabled with a non-flux gitopsEnforcement.reconciler.engine unless you
set reconciler.acknowledgeNoAuditLinkage=true, which turns linkage off
explicitly. See
Reconcile engine.
What audit linkage does¶
When Flux reconciles a change to an engineering CR (whose write was
approved via the ChangeRequest path), the
audit-flux-bridge component receives notification webhooks from Flux's
notification-controller. The kustomize-controller reports the apply
changeset (one Kind/namespace/name verb line per object) on its
Progressing events, while the authoritative revision only arrives on
the ReconciliationSucceeded event that follows. The bridge stashes the
changeset from the Progressing event and, when the matching success
event arrives, writes one AuditRecord per applied object, in that
object's own namespace (cluster-scoped objects land in the bridge
namespace, default dcs-system), plus a wrapper record for the
Kustomization itself in its namespace. Each record carries:
| Field | Source |
|---|---|
spec.gitCommitSHA |
The commit SHA Flux reconciled (extracted from event metadata) |
spec.gitPullRequestURL |
Looked up from the originating ChangeRequest's status.resolution.gitPullRequestURL |
spec.changeRequestRef |
<namespace>/<name> of the matching ChangeRequest |
spec.action |
Reconcile |
spec.target |
The applied object's kind + namespace + name. The changeset carries no apiVersion, so apiGroup is filled only when the kind resolves unambiguously in the bridge's scheme (all DCS kinds do); otherwise it is left empty |
spec.message |
Flux kustomize-controller reconciled <Kind>/<name> (<verb>) at <sha> — the verb (created / configured / deleted) comes from the changeset |
metadata.labels["dcs.io/audit-category"] |
flux-reconcile |
unchanged changeset entries are deliberately skipped, so Flux's
idempotent re-applies (the 10-minute heartbeat and interval reconciles)
do not spam per-object records. A per-(target, SHA) dedupe additionally
suppresses at-least-once webhook re-deliveries.
Together with the e-signature already recorded as an annotation on the target object (stamped by the change-control webhook at admission time), this produces an inspector-traceable chain from cluster state to Git commit to approving signature.
Pre-flight checklist¶
| Check | Why |
|---|---|
gitopsEnforcement.enabled=true |
The bridge only renders when GitOps enforcement is on |
Flux's notification-controller is installed |
The bridge receives events from this controller |
A Secret holding the HMAC shared secret exists in the chart's namespace |
Without it, anyone with network access to the bridge could inject fake events |
The cluster's Flux GitOps repo will receive a Provider + Alert pair |
These tell Flux where to send events |
The git-pr backend's repository is the one your site Kustomizations reconcile |
The backend edits the target's declaring file in place (#1325), so the existing site Kustomization delivers the merge — a repository Flux does not reconcile delivers nothing, and the smoke test below never fires |
The bridge does not require a ChangeRequest to exist for every
reconcile event. Events without a matching ChangeRequest still emit an
AuditRecord (with spec.changeRequestRef empty). This is intentional:
the audit trail must capture every reconcile, including direct Flux
applies that did not go through the propose-approve path.
Step-by-step¶
1. Create the shared-secret Secret¶
SHARED_SECRET=$(openssl rand -base64 32)
kubectl -n dcs-system create secret generic dcs-audit-flux-bridge-secret \
--from-literal=sharedSecret="$SHARED_SECRET"
Save $SHARED_SECRET somewhere durable. You'll need the same value on
the Flux side so signatures verify.
2. Stage the values change¶
# values-prod-audit-linkage.yaml
gitopsEnforcement:
enabled: true # already on
notification:
flux:
enabled: true
secretRef:
name: dcs-audit-flux-bridge-secret
# webhookEndpoint and listenPort retain chart defaults; override
# only if your cluster uses a non-default service mesh routing.
3. Helm upgrade¶
helm upgrade --reuse-values --install cloud-native-dcs \
oci://ghcr.io/cloud-native-dcs/charts/cloud-native-dcs \
--version <pinned-version> \
-f values-prod-audit-linkage.yaml
Confirm the bridge is running:
kubectl -n dcs-system get deploy,svc -l app.kubernetes.io/component=audit-flux-bridge
4. Configure Flux to send events to the bridge¶
On the cluster's Flux GitOps repo (the one Flux itself reconciles), add
a Provider + Alert pair. The chart's helm install NOTES print the
exact YAML, and the canonical form is:
---
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
name: dcs-audit-bridge
namespace: flux-system
spec:
type: generic-hmac
# Service name is fullname-prefixed (release "cloud-native-dcs" +
# "-audit-flux-bridge"). Adjust if your release name differs.
address: http://cloud-native-dcs-audit-flux-bridge.dcs-system.svc:61153/v1/flux-events
secretRef:
name: dcs-audit-flux-bridge-secret
---
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
name: dcs-audit-bridge
namespace: flux-system
spec:
providerRef:
name: dcs-audit-bridge
eventSeverity: info
eventSources:
- kind: Kustomization
name: '*'
The Secret referenced by Provider.spec.secretRef must live in the
flux-system namespace and have a key token whose value is the same
$SHARED_SECRET you provisioned in Step 1. Mirror it:
kubectl -n flux-system create secret generic dcs-audit-flux-bridge-secret \
--from-literal=token="$SHARED_SECRET"
5. Smoke-test the linkage¶
Trigger a Flux reconcile by editing any engineering CR via a
ChangeRequest and merging the resulting PR. Within ~10 seconds you
should see a fresh AuditRecord in the target namespace:
kubectl get auditrecords.audit.dcs.io -A \
-l audit.dcs.io/action=Reconcile \
--sort-by=.metadata.creationTimestamp
The record's spec.gitCommitSHA should match the merge commit on the
Flux source repo, and spec.changeRequestRef should point at the
originating ChangeRequest.
Inspector workflow¶
The whole point of this work is that an FDA inspector can walk the chain
from any artefact in either direction. Both directions are scriptable
with kubectl + jq:
Given an AuditRecord, find the originating Git commit and approving e-sig¶
Worked example below uses values captured from a reference cluster on
2026-05-11 (Reconcile of Kustomization/apps in flux-system against
commit bf1793cc of cndcs-deploy-demo):
NS=flux-system
NAME=audit-5x6kf
# 1. Pull the AuditRecord
kubectl -n "$NS" get auditrecord "$NAME" -o yaml
# 2. The Git commit is right there:
kubectl -n "$NS" get auditrecord "$NAME" \
-o jsonpath='{.spec.gitCommitSHA}'
# → bf1793cc4e9a214a8cf71957ea6b06bfc50bf1fd
# 3. The originating ChangeRequest (when one exists — Reconcile records
# for direct-Flux changes that bypassed the propose-only path will
# have an empty spec.changeRequestRef; that's expected):
CR_REF=$(kubectl -n "$NS" get auditrecord "$NAME" \
-o jsonpath='{.spec.changeRequestRef}')
if [ -n "$CR_REF" ]; then
CR_NS=${CR_REF%/*}
CR_NAME=${CR_REF#*/}
kubectl -n "$CR_NS" get changerequest "$CR_NAME" -o yaml
# 4. The e-signature is the dcs.io/esig-approved annotation on the
# target object, plus the canonical signer carried in the
# ChangeRequest's spec.signatures[]:
kubectl -n "$CR_NS" get changerequest "$CR_NAME" \
-o jsonpath='{.spec.signatures}' | jq .
fi
Given a Git commit, find the cluster artefacts it produced¶
SHA=bf1793cc4e9a214a8cf71957ea6b06bfc50bf1fd
# AuditRecords stamped with that SHA:
kubectl get auditrecords.audit.dcs.io -A -o json \
| jq --arg sha "$SHA" '.items[] | select(.spec.gitCommitSHA == $sha)'
# ChangeRequests whose merge produced that SHA:
kubectl get changerequests.changecontrol.dcs.io -A -o json \
| jq --arg sha "$SHA" '.items[] | select(.status.resolution.gitCommitSHA == $sha)'
Failure modes¶
| Symptom | Likely cause | Fix |
|---|---|---|
Bridge logs WARNING: starting WITHOUT HMAC verification |
secretRef.name is empty, OR the mounted Secret value is empty |
Set / rotate the shared secret; verify both Secrets carry 44 bytes (see "Rotating the shared secret" below). The bootstrap.sh re-run guard was added in cndcs-deploy-demo after cloud-native-dcs#326 to prevent empty values silently propagating. |
Notification-controller logs failed to dispatch notification: HMAC key is empty every interval |
Same as above — flux-system/dcs-audit-flux-bridge-secret token value is empty |
Rotate (see below) |
| 401 in bridge logs | Flux Provider's token value doesn't match the bridge's sharedSecret |
Re-mirror the secret across namespaces; both must be byte-identical |
AuditRecord lands but changeRequestRef is empty |
Reconcile didn't go through a ChangeRequest (direct Flux apply) |
Expected when a deployer pushes config that bypassed propose-only mode |
AuditRecord lands but gitCommitSHA is empty |
Flux event missing metadata.revision AND the involved object had no status-fallback SHA (e.g. HelmRelease backed by a HelmRepository or OCI chart) |
Expected for non-Git sources; not an error |
| Reconcile succeeds, bridge silent, no AuditRecord lands | Bridge accept-and-skips events whose metadata.revision is missing AND the involved object's status carries no resolvable SHA (return code 202 "missing revision"). As of cloud-native-dcs#340 the bridge backfills the SHA from Kustomization.status.lastAppliedRevision or, for HelmReleases, from the referenced GitRepository.status.artifact.revision, so this branch now fires only for HelmReleases with non-Git chart sources. |
Expected for non-Git Helm sources. For any other case, check the bridge logs for status-fallback GET ... failed (usually means missing RBAC on the Flux types). |
| Duplicate Reconcile records for one merge | Bridge dedupe TTL exceeded between re-deliveries | Default TTL is 30 min — re-tune if Flux notification-controller's at-least-once window is wider |
Wrapper Kustomization record lands but no per-object records |
The bridge restarted between the Progressing (changeset) event and the ReconciliationSucceeded event — the changeset stash is in-process — or more than 30 min elapsed between the two (stash TTL) |
Trigger a reconcile that actually changes the objects; unchanged entries never produce per-object records by design |
Rotating the shared secret¶
The HMAC shared secret lives in two namespaces:
dcs-system/dcs-audit-flux-bridge-secret, keysharedSecret— read by the bridge.flux-system/dcs-audit-flux-bridge-secret, keytoken— read by Flux'snotification-controllerwhen signing outbound events.
Both must hold the same value or HMAC verification fails (the bridge logs 401).
The bridge accepts events with a single secret value at any time, so rotation is two atomic edits with a brief mismatch window. To minimize the window, write the new value to both Secrets and then trigger a bridge restart so it re-reads the file:
NEW_SECRET=$(openssl rand -base64 32)
# 1. Update the bridge side first; the Deployment mounts the Secret as a
# file, so the bridge reloads when the Secret changes (the kubelet
# refreshes mounted Secrets within ~1 minute).
kubectl -n dcs-system create secret generic dcs-audit-flux-bridge-secret \
--from-literal=sharedSecret="$NEW_SECRET" \
--dry-run=client -o yaml | kubectl apply -f -
# 2. Update Flux's side. notification-controller re-reads the Secret on
# each event, so this takes effect immediately.
kubectl -n flux-system create secret generic dcs-audit-flux-bridge-secret \
--from-literal=token="$NEW_SECRET" \
--dry-run=client -o yaml | kubectl apply -f -
# 3. Force the bridge to pick the new value up immediately rather than
# waiting for the kubelet refresh window.
kubectl -n dcs-system rollout restart deployment \
-l app.kubernetes.io/component=audit-flux-bridge
# 4. Confirm the bridge logs no 401s on the next reconcile.
kubectl -n dcs-system logs -l app.kubernetes.io/component=audit-flux-bridge --tail=50
If the cluster is GitOps-managed (the Secrets are committed to Git), make the same edits in the deploy repo, and let Flux reconcile both Secrets on the same merge. That keeps both sides aligned.
Rotation does not invalidate prior AuditRecords. The records' integrity comes from the immutable etcd write. The HMAC only protects the delivery of the Flux event to the bridge.
Rollback¶
Set gitopsEnforcement.notification.flux.enabled=false and helm
upgrade. The Deployment, Service, RBAC, and the bridge image are
removed by Helm. Existing AuditRecord objects with Reconcile action
remain, immutable per §11.10(c).