Skip to content

PHON-226 + PHON-227: Scope for Pair Queries Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Close the two remaining scope gaps, which share one code shape. PHON-226: witness scope in /api/sentences covers pattern and CV-shape witnesses but not contrastive pair witnesses, so "Base forms only" + a minimal-pair rule can still witness on inflections. PHON-227: /api/contrastive has no scope support at all, so Contrast Sets ships a one-slot ScopeBar. Both are "apply scope predicates to both members of a word pair" — implemented once, used twice.

Architecture: A single worker-side helper emits scope SQL for an arbitrary word-column expression. /api/contrastive already proves the idiom for has_image (EXISTS (SELECT 1 FROM words w1 WHERE w1.word = p.word1 AND w1.has_image = 1) × both members), so the helper generalizes that rather than inventing a pattern. Both endpoints then gain the same opt-in flags, and Contrast Sets' ScopeBar grows back to three slots.

Tech Stack: Hono worker on D1 + vitest; React + TS + MUI frontend.

Spec: docs/superpowers/specs/2026-08-14-constraint-ui-unification-design.md §C (the Contrast Sets line struck in PHON-223 is restored by this work). Tickets PHON-226, PHON-227.

Global Constraints

  • Branch: feat/phon-226-227-pair-scope off develop. PR targets develop.
  • Opt-in only. Absent scope ⇒ byte-identical SQL and params on every touched query. Both endpoints' current default results must not move. This is the same invariant PHON-222 held and it is non-negotiable — pin it with exact-string tests per query builder.
  • Perf is the reason these were deferred. The pair self-joins are the heaviest path in sentence retrieval, and /api/contrastive fans out over pairs (642K rows). Task 4 is a real measurement, and its result decides whether this ships as-is — if a scoped query regresses badly, stop and report rather than shipping a slow path.
  • Values always bound; property names only from config constants (DEFAULT_SCOPE_EXCLUSIONS).
  • D1: 100 bind params per query — the scope bundle adds ≤6 per word-column reference, ×2 members = ≤12. Fine, but count them in the busiest query and say so.
  • Test commands: worker cd packages/web/workers && npx tsc --noEmit && npm test; frontend cd packages/web/frontend && npx tsc --noEmit && npm run lint && npm test && npm run build.
  • Commits end with: Co-Authored-By: Claude Fable 5 noreply@anthropic.com

Task 1: Shared pair-scope predicate helper

Files: - Modify: packages/web/workers/src/lib/wordFilter.ts (export the helper next to the existing scope logic — it is the natural home; if it fits better in config/properties.ts, say why and put it there) - Test: packages/web/workers/src/__tests__/registerFilter.test.ts or a new focused suite

Interfaces: - Produces:

export interface ScopeFlags {
  lemmas_only?: boolean;
  exclude_specialized?: boolean;
}

/** SQL predicates restricting a word to the requested scope, for an arbitrary
 *  column expression (`w1.word`, `p.word1`, `cs1.surface`, …). Returns an
 *  EXISTS-subquery fragment so it composes into any query that has a word
 *  reference, matching the idiom /api/contrastive already uses for has_image.
 *  Empty sql when no flags are set — callers must remain byte-identical. */
export function wordScopeExists(
  wordExpr: string,
  flags: ScopeFlags,
  alias: string,
): { sql: string; params: unknown[] };

Semantics: lemmas_only → (root IS NULL OR root = word); exclude_specialized → the NULL-safe DEFAULT_SCOPE_EXCLUSIONS bundle. The alias parameter avoids collisions when the helper is used twice in one query (w1/w2). Emit nothing when both flags are absent.

  • [ ] Step 1: Failing tests — no flags ⇒ empty sql and empty params; lemmas_only alone ⇒ one EXISTS with the root predicate and zero bound params; exclude_specialized alone ⇒ EXISTS with the three NULL-safe NOT INs and the six bundle values bound in order; both ⇒ one EXISTS carrying both; two calls with different aliases produce non-colliding SQL.
  • [ ] Step 2: Verify failure.
  • [ ] Step 3: Implement. Reuse DEFAULT_SCOPE_EXCLUSIONS; never interpolate values.
  • [ ] Step 4: Run the suite + full worker tests + tsc.
  • [ ] Step 5: Commit — feat(phon-226): shared pair-scope predicate helper

Task 2: PHON-226 — scope contrastive witnesses in /api/sentences

Files: - Modify: packages/web/workers/src/routes/sentences.ts (minpairMatchSql, maxoppMatchSql, multoppMatchSql, and the compileRules comments amended in PHON-222) - Test: packages/web/workers/src/routes/sentences.patterns.test.ts

Context to read first: minpairMatchSql (~line 225) and maxoppMatchSql (~line 256) both self-join corpus_sentences cs1 → pairs p → corpus_sentences cs2, so the two witness words are p.word1 and p.word2. multoppMatchSql follows below — scope it the same way if its witness columns are analogous; if its shape differs materially, scope what is analogous, say what you did, and don't force the rest.

Behavior: these builders take the scope flags and append wordScopeExists for BOTH pair members. Position and sonorant-diff clauses keep their current params order — append scope params in the position matching where the predicate text lands. Exclude-side semantics are unaffected (contrastive rules are include-side).

  • [ ] Step 1: Failing tests — for each builder: absent scope ⇒ SQL and params byte-identical to today (assert the full current literal, as PHON-222's guard test does); lemmas_only ⇒ both members carry the root predicate; exclude_specialized ⇒ both members carry the bundle with values bound; both flags compose; the position/sonorant clauses still bind in the right order (assert the full params array, not just membership).
  • [ ] Step 2: Verify failure.
  • [ ] Step 3: Implement, threading the flags from compileRules' body through to the three builders.
  • [ ] Step 4: Amend the PHON-222 comments in witnessScopeClause/compileRules that currently state contrastive witnesses are NOT scoped — they must now describe the true coverage. Leave no comment claiming a limitation that no longer exists.
  • [ ] Step 5: Run the suite + full worker tests + tsc. Commit — feat(phon-226): scope contrastive witnesses in sentence retrieval

Task 3: PHON-227 — scope support in /api/contrastive + Contrast Sets' full ScopeBar

Files: - Modify: packages/web/workers/src/routes/contrastive.ts (the five request bodies + their SQL branches) - Modify: packages/web/frontend/src/components/tools/ContrastiveInterventionTool.tsx - Modify: packages/web/frontend/src/services/ API client types as needed - Test: worker contrastive suite; ContrastiveInterventionTool.test.tsx

Worker: each endpoint's body type gains lemmas_only?: boolean and exclude_specialized?: boolean beside the existing has_image?: boolean; each SQL branch appends wordScopeExists for both pair members (or the single word, for the word-list branches around line 536 that filter words directly — there the predicate can be inlined rather than wrapped in EXISTS if that reads better; say which you did and why). Note the existing is_canonical = 1 hardcode stays — scope narrows further, it does not replace it.

Frontend: Contrast Sets' ScopeBar grows from one slot to three — Base forms and the specialized slot join Has image. Use the opt-in-restrict variant for the specialized slot (matching Sentences: Contrast Sets has no register default to drop, so ON = restrict, and today's results must not change). Keep the pair-semantics tooltip on Has image. Thread the two new flags into all five API calls.

  • [ ] Step 1: Failing tests — worker: per endpoint, absent flags ⇒ byte-identical SQL; each flag ⇒ predicates on both members; frontend: each toggle ON reaches the request body for at least one endpoint per mode, OFF omits it; the ScopeBar now renders three slots.
  • [ ] Step 2: Verify failure.
  • [ ] Step 3: Implement.
  • [ ] Step 4: Run both matrices. Commit — feat(phon-227): scope support in /api/contrastive + full ScopeBar in Contrast Sets

Task 4: Perf measurement (the gate)

Files: none — this is a measurement, reported not committed (except any index change it justifies).

Both tickets were deferred on perf grounds; this task discharges that.

  • [ ] Step 1: Build/refresh local D1 if needed so the measurement is against realistic data (~236K sentence index, 2.1M membership rows, 642K pairs). Say what data you measured against — a measurement on an empty DB is worthless and must be reported as such rather than presented as a result.
  • [ ] Step 2: For each of: sentences minpair, sentences maxopp, contrastive minimal-pairs, contrastive multiple-opposition — time the query unscoped (today's path) and with both scope flags, several runs each, reporting median and spread. Use wrangler d1 execute --local with timing, or an equivalent harness; state your method.
  • [ ] Step 3: Run EXPLAIN QUERY PLAN on the scoped variants and report whether the words lookups are index-backed (they join on word, which should be the primary key — confirm, don't assume).
  • [ ] Step 4: Verdict. If any scoped query regresses materially (say, >2× median on a realistic query), STOP and report rather than shipping — propose the index or query-shape change that would fix it. If the cost is negligible, say so with the numbers.
  • [ ] Step 5: Write the measurement into the PR body and, if an index was added, into the plan's record.

Task 5: Full matrix + push (PR after final review)

  • [ ] Step 1: worker tsc+tests; frontend tsc+lint+tests+build; python suite.
  • [ ] Step 2: confirm branch, git log --oneline origin/develop..HEAD, push. Never stage the pre-existing untracked repo-root files or uv.lock.
  • [ ] Step 3 (controller, after clean final review): PR to develop, including the perf numbers and noting that Contrast Sets' ScopeBar is now complete (the PHON-223 struck spec line is restored).

Self-Review Notes

  • Why one plan for two tickets: they are the same change (scope predicates on both members of a pair) against two endpoints. Splitting them would duplicate the helper, the tests, and the perf measurement.
  • The invariant to guard hardest: absent scope ⇒ byte-identical SQL on every touched builder. Two live surfaces depend on it (Sentences retrieval and all of Contrast Sets), and a drift here changes results silently rather than failing loudly.
  • Perf is a gate, not a formality (Task 4). These were explicitly deferred for measurement; shipping without it would defeat the deferral.
  • Known follow-on: with PHON-227 landed, Contrast Sets' ScopeBar reaches three slots and spec §C is fully satisfied for that tool. The remaining bespoke bits there (local state, hand-built display-only chips) stay as recorded in PHON-223 — out of scope here.