Skip to content

PHON-222: Sentences Aligned to the Shared Grammar Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Bring Sentences onto the shared grammar: the hardcoded six-norm one-sided bound picker is replaced by the metadata-driven PropertyFilterComposer, patterns/CV-shape/contrastive move onto the shared composers and PositionPicker, chips render through the one shared labeler, the three inconsistent chip-deletion mechanisms collapse into one, and a ScopeBar delivers a genuinely new capability — witness scope.

Architecture: Sentences keeps its own useConstraintStore instance (unchanged). Its four bespoke sections become thin arrangements of shared components. Witness scope is new worker behavior: /api/sentences' compileRules currently ignores scope entirely (verified — no lemmas_only/include_specialized/has_image handling), so scope constraints would silently drop. This PR teaches it to constrain the witness words — the words that satisfy include rules — leaving today's default results unchanged because both toggles are opt-in restrictions.

Tech Stack: Hono worker + vitest; React + TS + MUI, vitest + @testing-library/react.

Spec: docs/superpowers/specs/2026-08-14-constraint-ui-unification-design.md §C Sentences + §B grammar.

Global Constraints

  • Branch: feat/phon-222-sentences-grammar (created off develop at 0bcad295). PR targets develop.
  • Word Lists must be unaffected. Builder.tsx is not edited. Shared-component changes are additive and optional; Builder's call sites keep their current behavior byte-for-byte (its suites must stay green untouched).
  • Sentences' current default results must not change. Both scope toggles are opt-in restrictions: absent ⇒ today's SQL exactly. This is the user's explicit decision (2026-08-15) — sentence retrieval stays the full-vocabulary surface by default (CLAUDE.md).
  • Contrast Sets is out of scope (PHON-223).
  • Test commands: worker cd packages/web/workers && npx tsc --noEmit && npm test; frontend cd packages/web/frontend && npx tsc --noEmit && npm run lint && npm test && npm run build.
  • Commits end with: Co-Authored-By: Claude Fable 5 noreply@anthropic.com

Task 1: Worker — witness scope in compileRules (+ the exclude_specialized scope kind)

Files: - Modify: packages/web/workers/src/types.ts (ScopeConstraint kinds) - Modify: packages/web/workers/src/lib/constraintsToBody.ts (map the new kind) - Modify: packages/web/workers/src/lib/wordFilter.ts (honor the new kind — see below) - Modify: packages/web/workers/src/routes/sentences.ts (compileRules witness scope) - Test: packages/web/workers/src/routes/sentences.patterns.test.ts (extend), packages/web/workers/src/__tests__/constraintsToBody.test.ts (extend), packages/web/workers/src/__tests__/registerFilter.test.ts (extend)

Interfaces: - Consumes: existing ScopeConstraint { kind: 'lemmas_only' | 'include_specialized' }, DEFAULT_SCOPE_EXCLUSIONS, compileRules's MatchingWordsQuery shape. - Produces: - ScopeConstraint.kind gains 'exclude_specialized' — "apply DEFAULT_SCOPE_EXCLUSIONS to this query". On /api/words/search it is the standing default so the constraint is a harmless no-op; on /api/sentences it is the opt-in restriction. Mirrored into the frontend union in Task 2. - constraintsToBody maps it to body key exclude_specialized: true. - compileWordFilter honors body.exclude_specialized === true by applying the bundle even when an explicit categorical filter or include_specialized would otherwise suppress it — precedence rule: include_specialized wins if both are set (an explicit "show me everything" beats a redundant restriction request); pin this in a test. - compileRules applies witness scope to every include-side matching-words query it builds.

Witness-scope semantics (write this into the code comment): a scope constraint restricts which words may serve as witnesses for include rules — "Base forms only" means the word satisfying your pattern must be a base form (root IS NULL OR root = word); "Everyday vocabulary only" means it must not be in DEFAULT_SCOPE_EXCLUSIONS. Exclude-side rules are deliberately NOT scoped: an exclusion means "no word in the sentence may match", and narrowing which words count would weaken the exclusion into a loophole.

  • [ ] Step 1: Write the failing tests

constraintsToBody.test.ts:

describe('constraintsToBody: exclude_specialized scope', () => {
  it('maps the scope kind to the body flag', () => {
    const out = constraintsToBody([{ type: 'scope', kind: 'exclude_specialized' }] as Constraint[], base);
    expect((out as Record<string, unknown>).exclude_specialized).toBe(true);
  });
});

registerFilter.test.ts:

describe('exclude_specialized (opt-in bundle application)', () => {
  it('applies the bundle even when an explicit categorical filter would suppress it', () => {
    const compiled = compileWordFilter({ specialization: ['everyday'], exclude_specialized: true });
    const sql = compiled.wordsWhere.join(' AND ');
    expect(sql).toContain('w.specialization IN (?)');            // the explicit filter
    expect(sql).toContain('w.currency IS NULL OR w.currency NOT IN'); // bundle still applied
  });

  it('include_specialized wins when both are set', () => {
    const compiled = compileWordFilter({ include_specialized: true, exclude_specialized: true });
    const sql = compiled.wordsWhere.join(' AND ');
    expect(sql).not.toContain('w.currency IS NULL OR w.currency NOT IN');
  });
});

sentences.patterns.test.ts — the core of the task (follow the file's existing compileRules test idiom):

describe('compileRules witness scope', () => {
  it('absent scope leaves the SQL exactly as today', () => {
    // compileRules({ patterns: [...] }) — snapshot/compare includeMatches[0].sql
    // against the same call before scope existed: no root/specialization predicate
  });

  it('lemmas_only restricts witness words to base forms', () => {
    // includeMatches[0].sql contains '(root IS NULL OR root = word)'
  });

  it('exclude_specialized restricts witness words to the default scope bundle', () => {
    // includeMatches[0].sql contains the NULL-safe NOT IN predicates,
    // with the bundle values bound (not interpolated)
  });

  it('scope does NOT touch exclude-side matches (an exclusion must stay total)', () => {
    // body with an exclude pattern + lemmas_only:
    // excludeMatches[0].sql has NO root predicate
  });

  it('both scopes compose', () => {
    // lemmas_only + exclude_specialized → both predicates present on the include side
  });
});

  • [ ] Step 2: Run to verify failure — cd packages/web/workers && npx vitest run src/routes/sentences.patterns.test.ts src/__tests__/constraintsToBody.test.ts src/__tests__/registerFilter.test.ts.

  • [ ] Step 3: Implement

types.ts: extend the kind union + JSDoc explaining the asymmetric meaning (default on words/search, opt-in on sentences). constraintsToBody.ts: add the case. wordFilter.ts: in the default-scope block, compute const forceBundle = body.exclude_specialized === true and apply the bundle when (includeDefaultRegister && !wantsAll && !explicitFilter) || (forceBundle && !wantsAll) — keep the existing per-property suppression for the non-forced path.

sentences.ts — add a witness-scope helper near compileRules and apply it to every include-side query builder:

/** Witness scope (PHON-222). A scope constraint restricts which words may
 *  SERVE AS WITNESSES for include rules — "base forms only" means the word
 *  satisfying your pattern must be a base form. Exclude-side rules are NOT
 *  scoped: an exclusion means "no word in the sentence may match", and
 *  narrowing which words count would turn it into a loophole.
 *  Absent scope ⇒ byte-identical SQL to pre-PHON-222 (sentence retrieval is
 *  the full-vocabulary surface by default — CLAUDE.md). */
function witnessScopeClause(body: WordSearchBody): { sql: string; params: unknown[] } {
  const clauses: string[] = [];
  const params: unknown[] = [];
  if (body.lemmas_only === true) clauses.push('(root IS NULL OR root = word)');
  if (body.exclude_specialized === true) {
    for (const { property, values } of DEFAULT_SCOPE_EXCLUSIONS) {
      clauses.push(`(${property} IS NULL OR ${property} NOT IN (${values.map(() => '?').join(', ')}))`);
      params.push(...values);
    }
  }
  return { sql: clauses.length ? ` AND ${clauses.join(' AND ')}` : '', params };
}

Append scope.sql to each include-side SELECT word FROM words WHERE has_phonology = 1 AND (...) and append scope.params to that query's params in the correct positional order (the scope predicate goes after the existing predicate, so its params go last — verify against each builder). Do not touch exclude-side builders. Import DEFAULT_SCOPE_EXCLUSIONS from ../config/properties.

  • [ ] Step 4: Run the three suites + full worker suite + tsc.
  • [ ] Step 5: Commit — feat(phon-222): witness scope for sentence retrieval

Task 2: Frontend shared pieces — union mirror, entriesToChips subject, ScopeBar restrict-variant

Files: - Modify: packages/web/frontend/src/types/governance.ts (mirror the kind) - Modify: packages/web/frontend/src/lib/entriesToChips.ts (+ subject param) - Modify: packages/web/frontend/src/components/shared/ScopeBar.tsx (+ opt-in restrict variant) - Test: packages/web/frontend/src/lib/entriesToChips.test.ts, packages/web/frontend/src/components/shared/ScopeBar.test.tsx

Interfaces: - ScopeConstraint/StoreEntry scope kinds gain 'exclude_specialized' (mirror of Task 1). - entriesToChips(entries, propertyMap, removeAt, opts?: { subject?: 'word' | 'sentence' }) — subject: 'sentence' renders pattern chips in Sentences' existing voice (include → sentence has: /s/, sentence starts: /s/; exclude → no word has: /s/), matching today's constraintChips.ts output so Sentences users see no copy regression; 'word' (default) keeps Word Lists' current wording. Scope chips: exclude_specialized → Everyday vocabulary only; lemmas_only → Base forms only; include_specialized → Incl. specialized. - ScopeBar's specialized slot gains an optional variant?: 'default-restricted' | 'opt-in-restrict' (default 'default-restricted' = today's Word Lists behavior). 'opt-in-restrict' renders the slot as "Everyday vocabulary only" with tooltip "Only count everyday words as matches — excludes specialized, dated, and unassimilated vocabulary", and ON means restrict. This is a slot variant, not a fourth slot: the three-slot cap (spec §B) is unchanged.

  • [ ] Step 1: Failing tests — chip labels per subject (including the three scope kinds); ScopeBar renders the variant label + tooltip and reports ON as restriction; default variant unchanged (existing ScopeBar tests must pass untouched).
  • [ ] Step 2: Verify failure.
  • [ ] Step 3: Implement. Keep entriesToChips's existing signature backward-compatible (4th param optional) so Word Lists' call site needs no edit.
  • [ ] Step 4: Full frontend matrix. Builder's suites must pass unedited — that is the proof Word Lists is unaffected.
  • [ ] Step 5: Commit — feat(phon-222): chip subject voice + ScopeBar opt-in restrict variant

Task 3: Sentences rebuilt on the shared components

Files: - Modify: packages/web/frontend/src/components/tools/GovernedGenerationTool/index.tsx - Rewrite: PatternConstraints.tsx → thin PatternComposer wrapper (or delete and mount the composer directly in index) - Rewrite: PsycholinguisticsSection.tsx → PropertyFilterComposer wrapper - Modify: CvShapeSection.tsx (draft + include/exclude + Add, mirroring Word Lists) - Modify: ContrastiveSection.tsx (shared PositionPicker; drop its bespoke chip strip) - Delete: constraintChips.ts (superseded by entriesToChips) - Tests: update PatternConstraints.test.tsx, PsycholinguisticsSection.test.tsx, CvShapeSection.test.tsx; add an index-level test for the new surface

Behavior: - index.tsx: ScopeBar (Base forms + the opt-in-restrict specialized variant; no Has image — sentences are not picture cards) → the four sections → chips via entriesToChips(entries, propertyMap, removeAt, { subject: 'sentence' }). Scope state is local (like Word Lists) and travels as scope constraints appended at retrieve time, NOT as chips. - PsycholinguisticsSection: the hardcoded CURATED_BOUNDS array and its one-sided-threshold UI are deleted; the shared PropertyFilterComposer replaces them (metadata-driven, two-sided, percentile handling via the single shared rule). Bounds land through upsertBound so one chip per (norm, direction). - CvShapeSection: draft + include/exclude toggle + "Add shapes" (CV-shape exclusion is now honored by the sentences route — PHON-219 Task 1b), replacing the edit-in-place single entry. - ContrastiveSection: replace its bespoke position ToggleButtonGroup with the shared PositionPicker; delete its local chip strip (the constraints bar covers it). Keep the minpair/maxopp variants and the multopp exclusion comment as-is. - One deletion mechanism: every section removes via the store's removeAt (through the chips bar). Delete PatternConstraints' JSON.stringify-equality + load() removal, CvShapeSection's rebuild-the-list removal, and PsycholinguisticsSection's removeAt(entries.indexOf(e)). - Retrieve path: unchanged except that scope constraints are appended to the compiled list.

  • [ ] Step 1: Update/author the tests first (RED): pattern add via the shared composer; a two-sided bound reaching the request; CV-shape include and exclude; contrastive via PositionPicker; chips in sentence voice; scope toggles reaching the request as scope constraints; chip delete removes the entry.
  • [ ] Step 2: Verify failure.
  • [ ] Step 3: Implement, deleting the superseded code paths (do not leave dead exports).
  • [ ] Step 4: Full frontend matrix + a local smoke (worker + frontend dev; retrieve with a pattern, a bound, and Base forms ON; confirm results and the request body). Kill dev servers after.
  • [ ] Step 5: Commit — feat(phon-222): Sentences rebuilt on the shared grammar

Task 4: Full matrix + push (PR after final review)

  • [ ] Step 1: worker tsc+tests; frontend tsc+lint+tests+build; python suite (untouched, must be green).
  • [ ] Step 2: confirm branch, git log --oneline origin/develop..HEAD, push. Never stage the pre-existing untracked repo-root files or uv.lock.
  • [ ] Step 3 (controller, after clean final review): open the PR, documenting the new witness-scope semantics, that default results are unchanged, and the Sentences copy that moved.

Self-Review Notes

  • Spec §C Sentences coverage: metadata-driven two-sided bounds replacing CURATED_BOUNDS → T3; shared composers → T3; ScopeBar → T2 (variant) + T3 (mount); three deletion mechanisms → one → T3; the §C "verify the lemma predicate behaves sanely in sentence joins" item → T1, where verification found the route ignored scope entirely, escalated to the user, and became witness scope.
  • The user's decision (2026-08-15) — witness scope, opt-in, defaults unchanged — is why exclude_specialized exists as a separate kind rather than reusing include_specialized inverted: Sentences has no register default to drop, so "include specialized" is meaningless there, and inverting a shared toggle's meaning per tool is the kind of hidden inconsistency this workstream exists to remove.
  • Has image is deliberately absent from Sentences' ScopeBar: picture-card inventory says nothing about a sentence.
  • Known risk to watch in review: T1 appends scope params to existing queries — positional param order is the classic failure; every include-side builder must be checked individually, and the "absent scope ⇒ identical SQL" test is the guard.