Warden plan

28 - CR-23 (pending): Stance boundary audit: leak rate, shadow run, classifier fallback

← eval suite index


tier: hermetic requires: []


28 - CR-23 (pending): Stance boundary audit: leak rate, shadow run, classifier fallback

Pending stub — GATED criterion. The precondition below is not built; this plan reports the criterion as pending (skip, never fail) until it lands. When it does, replace this stub with a real plan in the shape of 25-cr-18 / 26-cr-25 / 27-cr-20.

Criterion implemented (verbatim from the register)

Register: ~/.studio/master.withrobin.ai/project/rfc.one/criteria-register.md (status there: PENDING OWNER ACCEPTANCE; accepted by the 2026-08-31 proxy screen).

### CR-23 — Stance boundary audit: leak rate, shadow run, classifier fallback - Statement: The provenance-drawn boundary (origin mark) is fine enough — wiki prose is not carrying genuine evidential conflict the exclusion suppresses, and the quoting-Entry leak stays within its priced residual. - Metric: Leak rate — fraction of quoting Entries producing at least one conflicts_with edge. Shadow run of the classifier over stance-bearing Signals (on-arm, edge-writing disabled); two annotators label each contradicts verdict as editorial disagreement or genuine evidential conflict (κ reported). - Dataset/inputs: 20 evidential Entries (meeting notes) per workspace, each quoting a wiki's stance with attribution, as the leak probe; the §11.2 on-arm shadow verdicts. - Threshold: Leak rate below 0.10 → the leak is the priced residual of a boundary drawn in provenance, reported as such. At or above 0.10 → the mark is too coarse in the quoting direction and the per-Signal stance classifier is built for that direction too. Redesign trigger: ≥20% of shadow contradicts verdicts labeled genuine evidential conflict by both annotators → the mark must be replaced by a per-Signal stance classifier at extraction time (a new trained component, charged as such against the §11 posture note — the single place a third trained component could enter). The classifier's bar, fixed in advance: precision ≥0.7 on its positive label (the anchor's §10.3 bar for the contradiction classifier, adopted because both work the region where de Marneffe et al. found precision scarcest), with the anchor's fallback ladder (stronger model, then human queue) behind it. - Drop condition: If the leak rate is at or above 0.10 and the classifier, when built, cannot hold the 0.7 precision bar, the perspectival half of the thesis fails (§1, §11.8): wiki prose leaves the extraction pipeline; wikis remain authored documents; the §4 formalization path dies. - Source: Companion §11.2 (leak named in §6; thesis consequence in §1 and §11.8). - Tier: gated — requires the Claim layer, contradiction check, origin mark.

Named precondition

Precondition: requires the Claim layer, the contradiction check, and the origin mark. Not built at HEAD 7f379974 (verified 2026-08-31: no 'claims', 'dimensions', or 'initiatives' tables in server/src/db/schema.ts, and no conflicts_with edges anywhere in server/src or packages/). The origin mark has no code marker to probe.

Skip guard

set -uo pipefail
source "$WARDEN_LIB/assert.sh"
cd "${PROJECT_ROOT:-$(git rev-parse --show-toplevel)}"

SCHEMA=server/src/db/schema.ts
MISSING=""
grep -q "'claims'" "$SCHEMA" || MISSING="$MISSING claims-tables"
grep -rq "conflicts_with" server/src packages 2>/dev/null || MISSING="$MISSING conflicts_with-edges"

if [ -n "$MISSING" ]; then
  warden_skip "CR-23 — Stance boundary audit: leak rate, shadow run, classifier fallback" "pending: requires the Claim layer, the contradiction check, and the origin mark — gated criterion, precondition not built (probes missing:$MISSING)"
else
  warden_skip "CR-23 — Stance boundary audit: leak rate, shadow run, classifier fallback" "schema probes now pass — upgrade this pending stub to a real plan before CR-23 can bind (probes are heuristic, not proof)"
fi

Shape (note for the next author)

Pending stub, hermetic on purpose: the guard reads the checkout only and emits skip in every branch, so this plan can never red a run — the criterion surfaces in every summary as pending with its precondition named. The probes are heuristic activation bells, not proof: quoted table names (and conflicts_with where the contradiction check is a precondition) flip the detail text when the machinery lands. The real plan must quote the criterion verbatim, turn its thresholds into warden_pass/warden_fail gates, and follow the conventions the three runnable plans set: machine-local fixtures (25-cr-18), NLI probe and versioned judge prompts (26-cr-25), data-keyed guard activation (27-cr-20).