Image Sense Gate Implementation Plan (PHON-205)¶

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Stop picture cards from depicting a sense the word doesn't carry, via a deterministic WordNet gate that re-runs on every future image tranche.

Architecture: A pure, injectable gate module (runtime/sense_gate.py) holds all WordNet logic and is unit-tested hermetically. Two developer-local scripts drive it: one resolves what each card depicts to a synset, the other applies the gate and writes a committed verdicts TSV. runtime/images.py gains only a loader + filter — it never imports WordNet, so the emit path stays dependency-light. Remediation (regenerate or blocklist) reuses the existing PHON-185 generation lane.

Tech Stack: Python 3.10+, polars, nltk (WordNet + SemCor), pytest. All build-time only; nothing new ships to D1 or the Worker.

Global Constraints¶

SemCor is build-time only, never committed. SemCor-derived counts cache to data/runtime/semcor_surface_senses.parquet, which is gitignored like the other runtime parquets. The committed verdicts artifact contains only words, WordNet synset ids, and verdicts — no corpus content. This mirrors the profanity-list precedent in packages/data/src/phonolex_data/runtime/images.py.
runtime/images.py must not import nltk. It reads a committed TSV. Keeping WordNet out of the emit path is a hard boundary.
abstain is a first-class verdict, distinct from compatible. Absence of evidence is never a flag. Expect ~45% abstain.
Three guards are mandatory (G1 corpus support, G2 derivational bridge, G3 abstain). Without them the rule over-flags 8× — 976/3,150 vs 120/3,150.
The derivational bridge must originate at the card synset, never at all senses of the card word. The looser form is what let rights pass in the prototype.
Tunable constants are pinned by tests, not by taste. MIN_SEMCOR_TOKENS, TOP_K, MAX_HOPS, MAX_DERIV_DEPTH all live at module top. The regression tests in Task 3 are the acceptance criteria; tune constants until they pass, do not weaken the tests.
Dominance is a PLURALITY, not a majority. (Corrected during execution — a MIN_SEMCOR_SHARE = 0.5 floor appears in Tasks 1/3 code below; it was deleted. It was never part of the prototype that measured 3.8%, and it abstained on 1,177 of 6,129 SemCor-covered shipped words (19.2%), concentrated in exactly the polysemous words the gate exists to catch.) A tie at the top abstains, however — 513 of 6,129 (8.4%) tie, and resolving by dict order is an artifact, not a signal.
Use .venv/bin/python, not uv run, for all Python commands in this repo.
Run all Python from the repo root. Test command: uv run python -m pytest packages/data/tests/ -v.
Reference: spec docs/superpowers/specs/2026-07-23-image-sense-gate-design.md; probes research/2026-07-23-phon-205-sense-gate/.

File Structure¶

File	Responsibility
`packages/data/src/phonolex_data/runtime/sense_gate.py`	Create. All WordNet logic: dominant-sense resolution, Mulberry card-synset resolution, compatibility relation, verdict. Pure functions, SemCor injected as a dict.
`packages/data/tests/runtime/test_sense_gate.py`	Create. Hermetic unit tests with a hand-written SemCor stub — no downloads.
`packages/data/scripts/build_semcor_cache.py`	Create. One-off: SemCor → gitignored parquet of surface→sense counts.
`packages/data/scripts/build_card_senses.py`	Create. Term 1 → `data/mappings/card_senses.tsv`. Mulberry deterministic; generated/openmoji filled by Task 7.
`packages/data/scripts/build_image_sense_gate.py`	Create. Applies the gate → `data/mappings/image_sense_verdicts.tsv`.
`packages/data/src/phonolex_data/runtime/images.py`	Modify. Load verdicts, drop `incompatible` pairs before selection and before inheritance.
`packages/data/src/phonolex_data/runtime/emit_parquet.py`	Modify. Plumb `image_verdicts_path` through to `build_word_images`.
`packages/data/tests/runtime/test_word_images.py`	Modify. Verdict-filter coverage + the `rights`/`cats` regressions.

Task 1: Dominant-sense resolution (G1 + G3)¶

Files: - Create: packages/data/src/phonolex_data/runtime/sense_gate.py - Test: packages/data/tests/runtime/test_sense_gate.py

Interfaces: - Consumes: nothing. - Produces: Dominant dataclass (synset: str, source: str); dominant_sense(word: str, semcor: SemcorCounts) -> Dominant | None. SemcorCounts = dict[str, dict[str, int]] mapping surface form → {synset name → count}.

[ ] Step 1: Write the failing test

Create packages/data/tests/runtime/test_sense_gate.py:

"""PHON-205 WordNet sense gate — hermetic unit coverage.

SemCor is injected as a plain dict so tests need no corpus download.
"""

import pytest

from phonolex_data.runtime.sense_gate import Dominant, dominant_sense

# Surface form -> {synset name: tagged count}. Hand-written; mirrors the real
# SemCor distribution for these words closely enough to pin behaviour.
SEMCOR = {
    "cat": {"cat.n.01": 9, "guy.n.01": 1},
    "rights": {"right.n.01": 12, "right.n.07": 1},
    "right": {"right.n.01": 26, "right.a.01": 19, "correct.a.01": 19},
    "papers": {"document.n.01": 13, "paper.n.01": 2, "newspaper.n.01": 2},
    "a": {"angstrom.n.01": 2},          # determiner 'a' is not sense-tagged
    "bikinis": {},
}


def test_semcor_majority_wins():
    d = dominant_sense("cat", SEMCOR)
    assert d == Dominant(synset="cat.n.01", source="semcor")


def test_semcor_surface_specific_beats_lemma_order():
    # WordNet ranks right.n.01 first for the lemma; SemCor confirms it for the
    # plural surface too. The point is that the SURFACE form is what is looked up.
    assert dominant_sense("rights", SEMCOR).synset == "right.n.01"


def test_abstains_below_token_floor():
    # 'a' has only 2 tagged tokens, all for a technical sense. Must not resolve
    # to angstrom.n.01 -- that false positive flagged the Alphabet card.
    assert dominant_sense("a", SEMCOR) is None


def test_abstains_when_no_confident_sense():
    # 'bikinis' has no SemCor evidence, and WordNet's rank-1 bikini.n.01
    # (Bikini Atoll) carries no tagged count.
    assert dominant_sense("bikinis", SEMCOR) is None


def test_abstains_for_word_absent_from_wordnet():
    assert dominant_sense("zzzznotaword", SEMCOR) is None


def test_falls_back_to_counted_wordnet_rank_one():
    # No SemCor entry, but WordNet's first sense has a nonzero tagged count.
    d = dominant_sense("dog", {})
    assert d is not None
    assert d.source == "wordnet-count"
    assert d.synset == "dog.n.01"

[ ] Step 2: Run test to verify it fails

Run: uv run python -m pytest packages/data/tests/runtime/test_sense_gate.py -v Expected: FAIL — ModuleNotFoundError: No module named 'phonolex_data.runtime.sense_gate'

[ ] Step 3: Write minimal implementation

Create packages/data/src/phonolex_data/runtime/sense_gate.py:

"""WordNet sense gate for picture cards (PHON-205).

A shipped card must not depict a sense the word does not carry. The gate
compares three terms:

  1. ``card_synset``  -- the sense the picture depicts (see ``mulberry_card_synset``
     for the deterministic Mulberry path; other providers are aligned offline)
  2. ``dominant_sense(word)`` -- the word's dominant sense, SemCor-first
  3. a graded compatibility relation between them

Design notes (see docs/superpowers/specs/2026-07-23-image-sense-gate-design.md):

* The bare comparison flags 31% of Mulberry pairs, overwhelmingly false
  positives. Three guards are mandatory -- G1 corpus support, G2 derivational
  bridge, G3 abstain -- bringing it to 3.8%.
* Term 1 must be a SYNSET, not a sense domain. At domain granularity the
  derivational bridge admits anything for a polysemous word.

This module holds all WordNet logic and is imported only by the build scripts.
``runtime.images`` reads the committed verdicts TSV and never imports nltk.
"""

from __future__ import annotations

from dataclasses import dataclass

from nltk.corpus import wordnet as wn

# --- G1 thresholds. Pinned by tests in tests/runtime/test_sense_gate.py. ---
# A surface form needs this many SemCor tokens before its distribution is
# trusted at all. 'a' has 2 tagged tokens (all angstrom) -- the determiner is
# not sense-tagged -- so the floor must sit above that.
MIN_SEMCOR_TOKENS = 3
# ...and the winning sense must hold at least this share of them.
MIN_SEMCOR_SHARE = 0.5

SemcorCounts = dict[str, dict[str, int]]


@dataclass(frozen=True)
class Dominant:
    """A word's dominant sense, with the evidence that established it."""

    synset: str
    source: str  # "semcor" | "wordnet-count"


def dominant_sense(word: str, semcor: SemcorCounts) -> Dominant | None:
    """Resolve ``word``'s dominant sense, or None to abstain (G1 + G3).

    SemCor is consulted first because it is indexed by SURFACE form, so it
    distinguishes `rights` from `right`; WordNet's sense order is lemma-indexed
    and cannot. WordNet's rank-1 sense is the fallback, but only when it carries
    a nonzero tagged count -- without that guard, rank 1 is polluted by rare and
    technical senses (bikini.n.01 is Bikini Atoll, bends is decompression
    sickness, b is bacillus), each of which produced a false flag.

    Returning None is a legitimate, common outcome (~45% of pairs). Absence of
    evidence must never become a flag.
    """
    synsets = wn.synsets(word)
    if not synsets:
        return None
    names = {s.name() for s in synsets}

    counts = semcor.get(word) or {}
    total = sum(counts.values())
    if total >= MIN_SEMCOR_TOKENS:
        best, n = max(counts.items(), key=lambda kv: kv[1])
        if n / total >= MIN_SEMCOR_SHARE and best in names:
            return Dominant(synset=best, source="semcor")

    top = synsets[0]
    if any(lemma.count() > 0 for lemma in top.lemmas() if lemma.name().lower() == word.lower()):
        return Dominant(synset=top.name(), source="wordnet-count")
    return None

[ ] Step 4: Run test to verify it passes

Run: uv run python -m pytest packages/data/tests/runtime/test_sense_gate.py -v Expected: PASS, 6 passed.

If test_abstains_below_token_floor fails, raise MIN_SEMCOR_TOKENS. Do not weaken the assertion — a resolving to angstrom is a confirmed false positive.

[ ] Step 5: Commit

git add packages/data/src/phonolex_data/runtime/sense_gate.py packages/data/tests/runtime/test_sense_gate.py
git commit -m "feat(phon-205): dominant-sense resolution with corpus-support and abstain guards"

Task 2: Mulberry card-synset resolution (term 1, deterministic)¶

Files: - Modify: packages/data/src/phonolex_data/runtime/sense_gate.py - Test: packages/data/tests/runtime/test_sense_gate.py

Interfaces: - Consumes: nothing from Task 1 (independent function in the same module). - Produces: CATEGORY_LEXNAMES: dict[str, set[str]]; mulberry_card_synset(word: str, symbol_category: str | None) -> str | None returning a synset name.

Mulberry's symbol_category (110 values in data/mappings/word_to_image.tsv) is a hand-assigned declaration of what the symbol depicts. Mapped to WordNet lexnames it constrains the word's synsets to one — no LLM needed for this provider.

[ ] Step 1: Write the failing test

Append to packages/data/tests/runtime/test_sense_gate.py:

from phonolex_data.runtime.sense_gate import mulberry_card_synset


def test_category_disambiguates_polysemous_word():
    # THE motivating case. 'right' has 8 noun senses; the Mulberry card is
    # categorised "Descriptive Direction", which selects the direction sense --
    # NOT right.n.01, the entitlement sense WordNet ranks first.
    assert mulberry_card_synset("right", "Descriptive Direction") == "right.n.02"


def test_category_selects_expected_sense_for_concrete_noun():
    assert mulberry_card_synset("cat", "Animal Mammal") == "cat.n.01"


def test_category_prefix_match_is_longest_first():
    # "Healthcare Body parts" must not fall through to the shorter "Healthcare".
    assert mulberry_card_synset("arm", "Healthcare Body parts") == "arm.n.01"


def test_unmapped_category_returns_none():
    assert mulberry_card_synset("cat", "Some Category We Never Mapped") is None


def test_no_matching_sense_returns_none():
    # Category is mapped, but the word has no sense in that domain. Note `cat`
    # DOES have noun.artifact senses (kat.n.01, caterpillar.n.02), so an
    # artifact-mapped category would wrongly resolve — noun.plant is genuinely
    # absent from `cat` and is the honest negative case.
    assert mulberry_card_synset("cat", "Plants and Trees") is None


def test_missing_category_returns_none():
    assert mulberry_card_synset("cat", None) is None

[ ] Step 2: Run test to verify it fails

Run: uv run python -m pytest packages/data/tests/runtime/test_sense_gate.py -v -k mulberry or category Expected: FAIL — ImportError: cannot import name 'mulberry_card_synset'

[ ] Step 3: Write minimal implementation

Append to packages/data/src/phonolex_data/runtime/sense_gate.py:

# ---------------------------------------------------------------------------
# Term 1 for the Mulberry stream — deterministic, no LLM.
# ---------------------------------------------------------------------------
# Mulberry's 110-value `symbol_category` taxonomy declares what each symbol
# depicts. Mapped to WordNet lexnames it narrows a word's synsets to the one the
# card shows. Matched by LONGEST PREFIX, so "Healthcare Body parts" wins over
# "Healthcare".

_ANY_VERB = {
    f"verb.{x}"
    for x in (
        "body change cognition communication competition consumption contact "
        "creation emotion motion perception possession social stative weather"
    ).split()
}
_ADJ = {"adj.all", "adj.pert", "adv.all"}

CATEGORY_LEXNAMES: dict[str, set[str]] = {
    "Verb": _ANY_VERB,
    "People Profession": {"noun.person"},
    "People Relationship": {"noun.person"},
    "People Descriptive": {"noun.person"} | _ADJ,
    "People Feelings": {"noun.feeling", "noun.state"} | _ADJ,
    "People": {"noun.person", "noun.group"},
    "Healthcare Body parts": {"noun.body"},
    "Healthcare Medical conditions": {"noun.state"},
    "Healthcare Grooming activities": _ANY_VERB | {"noun.act"},
    "Healthcare": {"noun.artifact", "noun.substance"},
    "Alphabet": {"noun.communication"},
    "Number Activity": {"noun.act"} | _ANY_VERB,
    "Number": {"noun.quantity", "noun.communication"},
    "Animal Habitat": {"noun.location", "noun.artifact"},
    "Animal Features": {"noun.body", "noun.animal"},
    "Animal": {"noun.animal"},
    "Clothes": {"noun.artifact"},
    "Food Kitchen items": {"noun.artifact"},
    "Food Kitchen actions": _ANY_VERB | {"noun.act"},
    "Food Feeding and eating": _ANY_VERB | {"noun.act", "noun.artifact"},
    "Food": {"noun.food", "noun.plant"},
    "Drink": {"noun.food"},
    "Plants and Trees": {"noun.plant"},
    "Descriptive Quantity": {"noun.quantity"} | _ADJ,
    "Descriptive Position": {"noun.location", "noun.relation", "noun.state"} | _ADJ,
    "Descriptive Direction": {"noun.location", "noun.relation"} | _ADJ,
    "Descriptive Time": {"noun.time"} | _ADJ,
    "Descriptive State": {"noun.state", "noun.attribute"} | _ADJ,
    "Descriptive Shape": {"noun.shape", "noun.attribute"} | _ADJ,
    "Descriptive": {"noun.attribute", "noun.state"} | _ADJ,
    "Transport": {"noun.artifact"},
    "Science Astronomy": {"noun.object", "noun.location"},
    "Science": {"noun.cognition", "noun.artifact", "noun.substance"},
    "Building": {"noun.artifact"},
    "Tools": {"noun.artifact"},
    "Sport": {"noun.act", "noun.artifact"},
    "Celebration Event": {"noun.event", "noun.act"},
    "Celebration": {"noun.artifact"},
    "Work and School Stationery": {"noun.artifact"},
    "Work and School": {"noun.cognition", "noun.act", "noun.artifact"},
    "Electrical": {"noun.artifact"},
    "Leisure": {"noun.artifact", "noun.act"},
    "Art Colour": {"noun.attribute"} | _ADJ,
    "Art": {"noun.artifact", "noun.act"},
    "Computer": {"noun.artifact", "noun.communication"},
    "Religion Person": {"noun.person"},
    "Religion Festival": {"noun.event", "noun.time"},
    "Religion": {"noun.cognition", "noun.act", "noun.artifact"},
    "Music": {"noun.artifact", "noun.act", "noun.communication"},
    "Holiday and travel": {"noun.act", "noun.artifact", "noun.location"},
    "Question": {"noun.communication"} | _ADJ,
}

_CATEGORIES_LONGEST_FIRST = sorted(CATEGORY_LEXNAMES, key=len, reverse=True)


def category_lexnames(symbol_category: str | None) -> set[str] | None:
    """Allowed WordNet lexnames for a Mulberry symbol_category, or None."""
    if not symbol_category:
        return None
    for prefix in _CATEGORIES_LONGEST_FIRST:
        if symbol_category.startswith(prefix):
            return CATEGORY_LEXNAMES[prefix]
    return None


def mulberry_card_synset(word: str, symbol_category: str | None) -> str | None:
    """Resolve which synset a Mulberry card depicts, or None if undeterminable.

    The category constrains the domain; among the word's synsets we take the
    highest-ranked one inside that domain. WordNet orders senses by frequency,
    so "highest-ranked in domain" is the most plausible reading of the category.
    """
    allowed = category_lexnames(symbol_category)
    if allowed is None:
        return None
    for synset in wn.synsets(word):
        if synset.lexname() in allowed:
            return synset.name()
    return None

[ ] Step 4: Run test to verify it passes

Run: uv run python -m pytest packages/data/tests/runtime/test_sense_gate.py -v Expected: PASS, 12 passed.

[ ] Step 5: Commit

git add packages/data/src/phonolex_data/runtime/sense_gate.py packages/data/tests/runtime/test_sense_gate.py
git commit -m "feat(phon-205): deterministic Mulberry card-synset resolution from symbol_category"

Task 3: Compatibility relation and verdict (G2 + tier asymmetry)¶

Files: - Modify: packages/data/src/phonolex_data/runtime/sense_gate.py - Test: packages/data/tests/runtime/test_sense_gate.py

Interfaces: - Consumes: Dominant, dominant_sense (Task 1). - Produces: Verdict dataclass (verdict: str ∈ {"compatible","incompatible","abstain"}, reason: str, card_synset: str | None, dominant_synset: str | None); gate_verdict(word: str, card_synset: str | None, tier: str, semcor: SemcorCounts) -> Verdict where tier ∈ {"strict","permissive"}.

This task carries the acceptance criteria for the whole gate. The regression tests below are non-negotiable; tune the constants until they pass.

[ ] Step 1: Write the failing test

Append to packages/data/tests/runtime/test_sense_gate.py:

from phonolex_data.runtime.sense_gate import Verdict, gate_verdict


def test_rights_is_incompatible_with_the_direction_card():
    """THE regression. `rights` inherits `right`'s Mulberry direction card.

    A prototype that compared sense DOMAINS rather than synsets let this pass,
    because `right`'s 8 noun senses have derivational forms spanning nearly
    every lexname. The bridge must originate at the card synset alone.
    """
    v = gate_verdict("rights", card_synset="right.n.02", tier="strict", semcor=SEMCOR)
    assert v.verdict == "incompatible"


def test_papers_is_incompatible_with_the_paper_sheet_card():
    # dominant(papers) = document.n.01; card depicts paper.n.01, a material.
    v = gate_verdict("papers", card_synset="paper.n.01", tier="strict", semcor=SEMCOR)
    assert v.verdict == "incompatible"


def test_identical_sense_is_compatible():
    v = gate_verdict("cat", card_synset="cat.n.01", tier="strict", semcor=SEMCOR)
    assert v.verdict == "compatible"


def test_derivational_shift_is_compatible():
    """`bicycling` (noun.act) inheriting the `bicycle` artifact card is CORRECT.

    A lexname shift reachable through derivational morphology is a
    morphological shift, not sense drift. Without this guard the gate flags
    every -ing and -ment nominalisation.
    """
    v = gate_verdict("bicycling", card_synset="bicycle.n.01", tier="strict", semcor={})
    assert v.verdict == "compatible"
    assert v.reason == "derivational"


def test_close_hypernym_is_compatible():
    # dog.n.01 vs its hypernym canine.n.02 -- same concept at another level.
    v = gate_verdict("dog", card_synset="canine.n.02", tier="strict", semcor={})
    assert v.verdict == "compatible"


def test_permissive_tier_accepts_a_non_dominant_but_top_k_sense():
    """Direct cards get top-K leniency, so `right` KEEPS its arrow.

    WordNet's frequency order comes from adult written corpora; direction is
    rank 2 for `right`. Strict scoring everywhere would strip child-dominant
    senses off cards that already passed PHON-185 review.
    """
    v = gate_verdict("right", card_synset="right.n.02", tier="permissive", semcor=SEMCOR)
    assert v.verdict == "compatible"


def test_strict_tier_rejects_what_permissive_accepts():
    v = gate_verdict("right", card_synset="right.n.02", tier="strict", semcor=SEMCOR)
    assert v.verdict == "incompatible"


def test_abstains_when_dominant_sense_is_unresolvable():
    v = gate_verdict("bikinis", card_synset="bikini.n.02", tier="strict", semcor=SEMCOR)
    assert v.verdict == "abstain"
    assert v.reason == "no-confident-sense"


def test_abstains_when_card_synset_is_unknown():
    v = gate_verdict("cat", card_synset=None, tier="strict", semcor=SEMCOR)
    assert v.verdict == "abstain"
    assert v.reason == "no-card-synset"


def test_rejects_unknown_tier():
    with pytest.raises(ValueError):
        gate_verdict("cat", card_synset="cat.n.01", tier="lenient", semcor=SEMCOR)

[ ] Step 2: Run test to verify it fails

Run: uv run python -m pytest packages/data/tests/runtime/test_sense_gate.py -v -k verdict or tier or compatible Expected: FAIL — ImportError: cannot import name 'gate_verdict'

[ ] Step 3: Write minimal implementation

Append to packages/data/src/phonolex_data/runtime/sense_gate.py:

# ---------------------------------------------------------------------------
# Compatibility relation + verdict.
# ---------------------------------------------------------------------------
# Equality would be far too strict: `papers` (document.n.01) vs a paper.n.01
# sheet-of-material card must flag, while `cats` vs cat.n.01 must not, and
# `bicycling` (noun.act) inheriting the bicycle artifact card must not.

# Hypernym/hyponym hops still counted as the same concept.
MAX_HOPS = 2
# Derivational chain depth from the CARD synset. bicycle.n.01 -> bicycle.v.01
# -> bicycling.n.01 needs 2.
MAX_DERIV_DEPTH = 2
# Sense-rank window for the permissive (direct-card) tier.
TOP_K = 3

_TIERS = ("strict", "permissive")


@dataclass(frozen=True)
class Verdict:
    verdict: str  # "compatible" | "incompatible" | "abstain"
    reason: str
    card_synset: str | None = None
    dominant_synset: str | None = None


def _taxonomically_close(a: str, b: str, max_hops: int = MAX_HOPS) -> bool:
    """True when two synsets are within ``max_hops`` of each other.

    ``shortest_path_distance`` returns None across parts of speech, which is the
    behaviour we want -- cross-POS closeness is the derivational guard's job.
    """
    if a == b:
        return True
    distance = wn.synset(a).shortest_path_distance(wn.synset(b))
    return distance is not None and distance <= max_hops


def _derivational_closure(name: str, depth: int = MAX_DERIV_DEPTH) -> set[str]:
    """Synsets reachable from ``name`` through derivational morphology.

    Rooted at the CARD synset, never at all senses of the card word -- the
    looser form reaches nearly every lexname for a polysemous word and admits
    anything (this is what let `rights` pass in the prototype).
    """
    seen = {name}
    frontier = {name}
    for _ in range(depth):
        nxt: set[str] = set()
        for current in frontier:
            for lemma in wn.synset(current).lemmas():
                for form in lemma.derivationally_related_forms():
                    target = form.synset().name()
                    if target not in seen:
                        nxt.add(target)
        seen |= nxt
        frontier = nxt
        if not frontier:
            break
    return seen


def gate_verdict(
    word: str,
    card_synset: str | None,
    tier: str,
    semcor: SemcorCounts,
) -> Verdict:
    """Decide whether ``card_synset`` is an acceptable depiction of ``word``.

    ``tier`` is "strict" for inherited cards (the sense must be dominant) and
    "permissive" for direct cards (any of the word's top-K senses will do).
    """
    if tier not in _TIERS:
        raise ValueError(f"tier must be one of {_TIERS}, got {tier!r}")
    if card_synset is None:
        return Verdict("abstain", "no-card-synset")

    dominant = dominant_sense(word, semcor)
    if dominant is None:
        return Verdict("abstain", "no-confident-sense", card_synset=card_synset)

    common = dict(card_synset=card_synset, dominant_synset=dominant.synset)

    if card_synset == dominant.synset:
        return Verdict("compatible", "exact", **common)
    if _taxonomically_close(card_synset, dominant.synset):
        return Verdict("compatible", "taxonomic", **common)
    if dominant.synset in _derivational_closure(card_synset):
        return Verdict("compatible", "derivational", **common)

    if tier == "permissive":
        top_k = {s.name() for s in wn.synsets(word)[:TOP_K]}
        if card_synset in top_k:
            return Verdict("compatible", "top-k", **common)

    return Verdict("incompatible", f"{tier}-mismatch", **common)

[ ] Step 4: Run test to verify it passes

Run: uv run python -m pytest packages/data/tests/runtime/test_sense_gate.py -v Expected: PASS, 22 passed.

If test_rights_is_incompatible_with_the_direction_card fails, the derivational closure is too wide — lower MAX_DERIV_DEPTH to 1 and re-check that test_derivational_shift_is_compatible still passes. If both cannot hold at once, restrict the closure to same-POS targets. Do not weaken either test.

[ ] Step 5: Commit

git add packages/data/src/phonolex_data/runtime/sense_gate.py packages/data/tests/runtime/test_sense_gate.py
git commit -m "feat(phon-205): compatibility relation with derivational bridge and tier asymmetry"

Task 4: SemCor cache + card-senses artifact¶

Files: - Create: packages/data/scripts/build_semcor_cache.py - Create: packages/data/scripts/build_card_senses.py - Modify: .gitignore

Interfaces: - Consumes: mulberry_card_synset (Task 2). - Produces: data/runtime/semcor_surface_senses.parquet (gitignored) with columns surface, synset, count; data/mappings/card_senses.tsv (committed) with columns word, provider, image_file, card_synset, source_signal. Also load_semcor_counts(path) -> SemcorCounts in build_semcor_cache.py, imported by Task 5.

Corrected during execution: build_card_senses.py must read two mapping files. word_to_image.tsv holds only Mulberry + OpenMoji (3,649 rows); the generated stream lives in generated_word_images.tsv (9,724 rows) and its image_file already carries the .webp extension. Omitting it makes the gate silently abstain on ~83% of shipped cards and leaves Task 7 with an empty worklist. Expected total: 13,373 rows — mulberry 1,510 resolved / 190 unmapped, generated 9,724 and openmoji 1,949 needs-alignment.

[ ] Step 1: Add the gitignore entry

Append to .gitignore:

# PHON-205: SemCor-derived surface->sense counts. Build-time only; SemCor
# content is never committed or shipped (see the sense-gate spec).
data/runtime/semcor_surface_senses.parquet

[ ] Step 2: Write the SemCor cache builder

Create packages/data/scripts/build_semcor_cache.py:

"""Cache SemCor surface-form sense counts (PHON-205).

SemCor is indexed by SURFACE form, so it distinguishes `rights` from `right`;
WordNet's sense ordering is lemma-indexed and cannot. The gate consults it first
when resolving a word's dominant sense.

The cache is a GITIGNORED build product. SemCor content is never committed and
never ships -- only WordNet synset ids reach the committed verdicts artifact.

Usage:
    uv run python packages/data/scripts/build_semcor_cache.py
"""

from __future__ import annotations

from collections import Counter, defaultdict
from pathlib import Path

import nltk
import polars as pl

REPO_ROOT = Path(__file__).resolve().parents[3]
CACHE_PATH = REPO_ROOT / "data/runtime/semcor_surface_senses.parquet"


def load_semcor_counts(path: Path = CACHE_PATH) -> dict[str, dict[str, int]]:
    """Read the cache into the shape ``sense_gate`` expects."""
    if not path.exists():
        raise SystemExit(
            f"{path} missing — run: uv run python packages/data/scripts/build_semcor_cache.py"
        )
    out: dict[str, dict[str, int]] = defaultdict(dict)
    for row in pl.read_parquet(path).iter_rows(named=True):
        out[row["surface"]][row["synset"]] = row["count"]
    return dict(out)


def main() -> None:
    for corpus in ("wordnet", "semcor"):
        nltk.download(corpus, quiet=True)
    from nltk.corpus import semcor

    counts: dict[str, Counter] = defaultdict(Counter)
    for tree in semcor.tagged_sents(tag="sem"):
        for chunk in tree:
            label = getattr(chunk, "label", None)
            if label is None:
                continue
            lemma = chunk.label()
            synset = getattr(lemma, "synset", None)
            if synset is None:
                continue
            surface = " ".join(chunk.leaves()).lower()
            counts[surface][lemma.synset().name()] += 1

    rows = [
        {"surface": surface, "synset": synset, "count": n}
        for surface, c in counts.items()
        for synset, n in c.items()
    ]
    CACHE_PATH.parent.mkdir(parents=True, exist_ok=True)
    pl.DataFrame(rows).write_parquet(CACHE_PATH, compression="zstd")
    print(f"wrote {CACHE_PATH} ({len(rows):,} rows, {len(counts):,} surfaces)")


if __name__ == "__main__":
    main()

[ ] Step 3: Run it and verify the cache

Run:

uv run python packages/data/scripts/build_semcor_cache.py
uv run python -c "
from pathlib import Path
import sys; sys.path.insert(0, 'packages/data/scripts')
from build_semcor_cache import load_semcor_counts
c = load_semcor_counts()
print('surfaces:', len(c))
print('rights  :', c.get('rights'))
print('right   :', c.get('right'))
"

Expected: ~31,000 surfaces; rights shows {'right.n.01': 12, 'right.n.07': 1}; right shows a spread including right.n.01.

[ ] Step 4: Write the card-senses builder

Create packages/data/scripts/build_card_senses.py:

"""Emit data/mappings/card_senses.tsv — term 1 of the sense gate (PHON-205).

"Term 1" is the sense each shipped picture card depicts, as a WordNet synset.

  * mulberry  -- DETERMINISTIC. The 110-value `symbol_category` taxonomy in
                 word_to_image.tsv is a hand-assigned sense declaration; mapped
                 to lexnames it disambiguates the word's synsets. No LLM.
  * generated -- from the PHON-185 cardspec gloss; filled by the alignment pass.
  * openmoji  -- from the annotation; filled by the alignment pass.

Rows this script cannot resolve are emitted with an empty ``card_synset`` so the
artifact always lists every shipped card and the alignment pass has an explicit
worklist.

Usage:
    uv run python packages/data/scripts/build_card_senses.py
"""

from __future__ import annotations

import csv
from pathlib import Path

import polars as pl

from phonolex_data.runtime.sense_gate import mulberry_card_synset

REPO_ROOT = Path(__file__).resolve().parents[3]
WORD_IMAGE_TSV = REPO_ROOT / "data/mappings/word_to_image.tsv"
OUT_TSV = REPO_ROOT / "data/mappings/card_senses.tsv"
FIELDS = ["word", "provider", "image_file", "card_synset", "source_signal"]


def main() -> None:
    mapping = pl.read_csv(
        WORD_IMAGE_TSV,
        separator="\t",
        schema_overrides={"word": pl.Utf8, "image_id": pl.Utf8, "variant": pl.Utf8},
    )

    rows: list[dict[str, str]] = []
    for r in mapping.iter_rows(named=True):
        word, provider = r["word"], r["source"]
        synset = ""
        signal = ""
        if provider == "mulberry":
            resolved = mulberry_card_synset(word, r.get("symbol_category"))
            if resolved:
                synset, signal = resolved, "symbol_category"
            else:
                signal = "unmapped-category"
        else:
            signal = "needs-alignment"
        rows.append(
            {
                "word": word,
                "provider": provider,
                "image_file": f"{r['image_id']}.svg",
                "card_synset": synset,
                "source_signal": signal,
            }
        )

    OUT_TSV.parent.mkdir(parents=True, exist_ok=True)
    with OUT_TSV.open("w", newline="") as fh:
        w = csv.DictWriter(fh, fieldnames=FIELDS, delimiter="\t")
        w.writeheader()
        w.writerows(rows)

    resolved = sum(1 for r in rows if r["card_synset"])
    mulberry = sum(1 for r in rows if r["provider"] == "mulberry")
    mul_resolved = sum(1 for r in rows if r["provider"] == "mulberry" and r["card_synset"])
    print(f"wrote {OUT_TSV} ({len(rows):,} rows, {resolved:,} with a synset)")
    print(f"  mulberry: {mul_resolved:,}/{mulberry:,} resolved deterministically")


if __name__ == "__main__":
    main()

[ ] Step 5: Run it and sanity-check the motivating row

Run:

uv run python packages/data/scripts/build_card_senses.py
grep -P '^right\tmulberry' data/mappings/card_senses.tsv

Expected: the right row carries right.n.02 and symbol_category. Mulberry resolution should land near 80–90%; a much lower figure means CATEGORY_LEXNAMES needs more entries.

[ ] Step 6: Commit

git add .gitignore packages/data/scripts/build_semcor_cache.py packages/data/scripts/build_card_senses.py data/mappings/card_senses.tsv
git commit -m "feat(phon-205): SemCor cache + deterministic Mulberry card-sense artifact"

Task 5: Verdicts artifact + review checkpoint¶

Files: - Create: packages/data/scripts/build_image_sense_gate.py

Interfaces: - Consumes: gate_verdict (Task 3), load_semcor_counts (Task 4), data/mappings/card_senses.tsv (Task 4). - Produces: data/mappings/image_sense_verdicts.tsv with columns word, provider, image_file, tier, card_synset, dominant_synset, verdict, reason.

The tier is derived from the shipped match_type: root_inherited → strict, everything else → permissive. An inherited card's synset is looked up under its root, because that is the word the image belongs to.

[ ] Step 1: Write the script

Create packages/data/scripts/build_image_sense_gate.py:

"""Emit data/mappings/image_sense_verdicts.tsv — the sense gate (PHON-205).

Applies the WordNet gate to every shipped picture card and writes a committed,
human-reviewable verdict per (word, image) pair. The rule PROPOSES; nothing is
auto-applied. `runtime.images` drops only rows this file marks `incompatible`.

Tiers are asymmetric: inherited cards are scored strictly (the card's sense must
be the word's dominant sense) while direct cards get top-K leniency, because
WordNet's frequency order reflects adult written register and would otherwise
strip child-dominant senses off reviewed cards.

Usage:
    uv run python packages/data/scripts/build_semcor_cache.py    # once
    uv run python packages/data/scripts/build_card_senses.py     # once
    uv run python packages/data/scripts/build_image_sense_gate.py
"""

from __future__ import annotations

import csv
import re
import sys
from collections import Counter
from pathlib import Path

import polars as pl
from nltk.corpus import wordnet as wn

REPO_ROOT = Path(__file__).resolve().parents[3]
sys.path.insert(0, str(Path(__file__).resolve().parent))

from build_semcor_cache import load_semcor_counts  # noqa: E402

from phonolex_data.runtime.sense_gate import gate_verdict  # noqa: E402

WORD_IMAGES_PARQUET = REPO_ROOT / "data/runtime/word_images.parquet"
WORDS_PARQUET = REPO_ROOT / "data/runtime/words.parquet"
CARD_SENSES_TSV = REPO_ROOT / "data/mappings/card_senses.tsv"
OUT_TSV = REPO_ROOT / "data/mappings/image_sense_verdicts.tsv"

FIELDS = [
    "word", "provider", "image_file", "tier",
    "card_synset", "dominant_synset", "verdict", "reason", "supporting_signals",
]

# --- Review aids ------------------------------------------------------------
# These do NOT change verdicts. They annotate a flag with corroborating lexical
# evidence so the human reviewer can triage. Each is gated to the surface class
# where it is not noise: ungated, own-lemma disjointness fires on participial
# adjectives (`abandoned`, `accepted`) and SemCor divergence fires on verb tense
# (`made`, `told`, `asked`), which is why neither works as a detector alone.

_PLURAL_MARKER = re.compile(
    r"\(\s*(frequently |usually |often |always |chiefly )?plural", re.I
)


def _is_plural_surface(word: str, root: str | None) -> bool:
    return bool(root) and word != root and word.endswith("s") and not word.endswith("ss")


def _own_synsets(word: str) -> set[str]:
    return {
        s.name()
        for s in wn.synsets(word)
        if any(lem.name().lower() == word.lower() for lem in s.lemmas())
    }


def supporting_signals(
    word: str, root: str | None, semcor: dict[str, dict[str, int]]
) -> list[str]:
    """Corroborating evidence for a flag. Advisory only."""
    signals: list[str] = []
    if not root:
        return signals
    plural = _is_plural_surface(word, root)

    if plural and any(
        _PLURAL_MARKER.search(s.definition() or "") for s in wn.synsets(root, "n")
    ):
        signals.append("plural-marker")

    if plural:
        own, of_root = _own_synsets(word), _own_synsets(root)
        if own and not (own & of_root):
            signals.append("own-lemma-disjoint")

    counts_word, counts_root = semcor.get(word) or {}, semcor.get(root) or {}
    if counts_word and counts_root:
        top_word = max(counts_word.items(), key=lambda kv: kv[1])[0]
        top_root = max(counts_root.items(), key=lambda kv: kv[1])[0]
        if top_word != top_root and top_word.split(".")[1] == "n":
            signals.append("semcor-divergent")

    return signals


def main() -> None:
    semcor = load_semcor_counts()

    card_senses = pl.read_csv(
        CARD_SENSES_TSV, separator="\t", schema_overrides={"word": pl.Utf8, "card_synset": pl.Utf8}
    )
    # (word, provider, image_file) -> card synset.
    #
    # image_file MUST be part of the key. 56 Mulberry words carry more than one
    # distinct symbol_category across their candidate rows, so the same word can
    # legitimately resolve to different synsets for different cards. Keying on
    # (word, provider) alone lets a dict comprehension silently overwrite, and
    # the gate would then compare the shipped card against some *other* card's
    # sense — producing a false drop or a false pass.
    synset_of: dict[tuple[str, str, str], str] = {
        (r["word"], r["provider"], r["image_file"]): r["card_synset"]
        for r in card_senses.iter_rows(named=True)
        if r["card_synset"]
    }

    shipped = pl.read_parquet(WORD_IMAGES_PARQUET)
    roots = pl.read_parquet(WORDS_PARQUET).select("word", "root")
    shipped = shipped.join(roots, on="word", how="left")

    rows: list[dict[str, str]] = []
    for r in shipped.iter_rows(named=True):
        inherited = r["match_type"] == "root_inherited"
        tier = "strict" if inherited else "permissive"
        # An inherited card belongs to the ROOT — look its sense up there.
        # `image_file` is carried over unchanged by root inheritance (see
        # images.py's inherited join), so it keys correctly for both tiers.
        source_word = r["root"] if inherited else r["word"]
        card_synset = synset_of.get((source_word or "", r["provider"], r["image_file"]))

        v = gate_verdict(r["word"], card_synset, tier, semcor)
        signals = (
            supporting_signals(r["word"], r["root"], semcor)
            if v.verdict == "incompatible"
            else []
        )
        rows.append(
            {
                "word": r["word"],
                "provider": r["provider"],
                "image_file": r["image_file"],
                "tier": tier,
                "card_synset": v.card_synset or "",
                "dominant_synset": v.dominant_synset or "",
                "verdict": v.verdict,
                "reason": v.reason,
                "supporting_signals": ",".join(signals),
            }
        )

    OUT_TSV.parent.mkdir(parents=True, exist_ok=True)
    with OUT_TSV.open("w", newline="") as fh:
        w = csv.DictWriter(fh, fieldnames=FIELDS, delimiter="\t")
        w.writeheader()
        w.writerows(rows)

    tally = Counter(r["verdict"] for r in rows)
    by_tier = Counter((r["tier"], r["verdict"]) for r in rows)
    print(f"wrote {OUT_TSV} ({len(rows):,} rows)")
    for verdict, n in tally.most_common():
        print(f"  {verdict:13s} {n:6,}  ({100 * n / len(rows):5.1f}%)")
    print("  by tier:")
    for (tier, verdict), n in sorted(by_tier.items()):
        print(f"    {tier:11s} {verdict:13s} {n:6,}")


if __name__ == "__main__":
    main()

[ ] Step 2: Run it

Run: uv run python packages/data/scripts/build_image_sense_gate.py Expected: ~22,044 rows. Mulberry incompatible should land near the prototype's 3.8%; a rate above ~10% means a guard regressed.

[ ] Step 3: Verify the motivating cases

Run:

grep -P '^(rights|right|papers|paper|cats|cat)\t' data/mappings/image_sense_verdicts.tsv

Expected: rights → incompatible; right → compatible; papers → incompatible; cat/cats → compatible.

If rights is not incompatible, stop and fix the gate before continuing. That is the ticket.

[ ] Step 4: CHECKPOINT — human review

Report to the user before proceeding: - total incompatible, split by tier and provider - how many incompatible words have no other provider (they go imageless) - a sample of 30 incompatible rows for eyeballing

Do not commit verdicts the user has not seen. This is the PHON-197 pattern: the rule proposes, a human confirms.

[ ] Step 5: Commit

git add packages/data/scripts/build_image_sense_gate.py data/mappings/image_sense_verdicts.tsv
git commit -m "feat(phon-205): image sense verdicts artifact"

Task 6: Wire the verdicts into the shipped mapping¶

Files: - Modify: packages/data/src/phonolex_data/runtime/images.py - Modify: packages/data/src/phonolex_data/runtime/emit_parquet.py:164-217 - Test: packages/data/tests/runtime/test_word_images.py

Interfaces: - Consumes: data/mappings/image_sense_verdicts.tsv (Task 5). - Produces: load_sense_verdicts(path) -> set[tuple[str, str, str]] (word, provider, image_file triples marked incompatible); build_word_images(..., verdicts_path: Path | None = None).

images.py must not import nltk. It reads the TSV only.

[ ] Step 1: Write the failing test

Append to packages/data/tests/runtime/test_word_images.py:

_VERDICT_HEADER = (
    "word\tprovider\timage_file\ttier\tcard_synset\tdominant_synset\tverdict\treason"
)


@pytest.fixture
def verdicts_path(tmp_path: Path) -> Path:
    p = tmp_path / "image_sense_verdicts.tsv"
    p.write_text(
        "\n".join(
            [
                _VERDICT_HEADER,
                # cat's own card is fine
                "cat\tmulberry\tcat.svg\tpermissive\tcat.n.01\tcat.n.01\tcompatible\texact",
                # cats inherits it, and that is also fine
                "cats\tmulberry\tcat.svg\tstrict\tcat.n.01\tcat.n.01\tcompatible\texact",
                # book's own card is sense-wrong -> must be dropped
                "book\topenmoji\t1F4D6.svg\tpermissive\tbook.n.02\tbook.n.11"
                "\tincompatible\tpermissive-mismatch",
                # abstain must NOT drop anything
                "hairbrush\tmulberry\thair_brush.svg\tpermissive\t\t\tabstain"
                "\tno-confident-sense",
            ]
        )
        + "\n"
    )
    return p


def test_incompatible_card_is_dropped(mapping_paths, verdicts_path):
    mapping, review = mapping_paths
    df = build_word_images(
        _words_df([("cat", None), ("book", None), ("hairbrush", None)]),
        mapping_path=mapping,
        review_path=review,
        verdicts_path=verdicts_path,
    )
    shipped = set(df.get_column("word"))
    assert "cat" in shipped
    assert "book" not in shipped          # incompatible
    assert "hairbrush" in shipped         # abstain must not drop


def test_dropped_root_card_is_not_inherited(mapping_paths, verdicts_path):
    """A root whose card is dropped must not pass it on. This is the fix.

    `books` has no direct card; without the gate it would inherit `book`'s.
    """
    mapping, review = mapping_paths
    df = build_word_images(
        _words_df([("book", None), ("books", "book")]),
        mapping_path=mapping,
        review_path=review,
        verdicts_path=verdicts_path,
    )
    assert "books" not in set(df.get_column("word"))


def test_compatible_inheritance_still_works(mapping_paths, verdicts_path):
    mapping, review = mapping_paths
    df = build_word_images(
        _words_df([("cat", None), ("cats", "cat")]),
        mapping_path=mapping,
        review_path=review,
        verdicts_path=verdicts_path,
    )
    row = df.filter(pl.col("word") == "cats").to_dicts()[0]
    assert row["match_type"] == ROOT_INHERITED
    assert row["image_file"] == "cat.svg"


def test_no_verdicts_path_is_a_no_op(mapping_paths):
    """Omitting verdicts must leave behaviour identical — hermetic fixtures rely on it."""
    mapping, review = mapping_paths
    df = build_word_images(
        _words_df([("cat", None), ("book", None)]),
        mapping_path=mapping,
        review_path=review,
    )
    assert "book" in set(df.get_column("word"))

[ ] Step 2: Run test to verify it fails

Run: uv run python -m pytest packages/data/tests/runtime/test_word_images.py -v -k verdict or dropped or inheritance Expected: FAIL — TypeError: build_word_images() got an unexpected keyword argument 'verdicts_path'

[ ] Step 3: Implement the loader and the filter

In packages/data/src/phonolex_data/runtime/images.py, add the path constant next to the existing ones:

# PHON-205: committed sense-gate verdicts. Rows marked `incompatible` are
# dropped from the shipped pool before selection AND before root inheritance,
# so a sense-wrong card can never be passed to an inflection. Built by
# packages/data/scripts/build_image_sense_gate.py. This module deliberately does
# NOT import nltk — the WordNet logic lives in runtime/sense_gate.py.
SENSE_VERDICTS_TSV = REPO_ROOT / "data/mappings/image_sense_verdicts.tsv"

Add the loader after load_blocklist:

def load_sense_verdicts(path: Path = SENSE_VERDICTS_TSV) -> set[tuple[str, str, str]]:
    """Load the (word, provider, image_file) triples marked ``incompatible``.

    Only ``incompatible`` is actionable. ``abstain`` means the gate had no
    confident dominant sense and must leave the card alone — it is not a
    soft reject.
    """
    v = pl.read_csv(
        path,
        separator="\t",
        schema_overrides={"word": pl.Utf8, "image_file": pl.Utf8, "verdict": pl.Utf8},
    ).filter(pl.col("verdict") == "incompatible")
    return {(r["word"], r["provider"], r["image_file"]) for r in v.iter_rows(named=True)}

Change the build_word_images signature to add the parameter after blocklist_path:

    verdicts_path: Path | None = None,

And insert the filter immediately after the blocklist filter, before the lexicon semi-join:

    # Sense gate (PHON-205): drop cards that depict a sense the word does not
    # carry. Applied to the shipped pool so `inherited` (which draws only from
    # `direct`) cannot propagate a dropped card.
    if verdicts_path is not None:
        incompatible = load_sense_verdicts(verdicts_path)
        if incompatible:
            drop = pl.DataFrame(
                [
                    {"word": w, "provider": p, "image_file": f}
                    for w, p, f in sorted(incompatible)
                ],
                schema={"word": pl.Utf8, "provider": pl.Utf8, "image_file": pl.Utf8},
            )
            shipped = shipped.join(
                drop, on=["word", "provider", "image_file"], how="anti"
            )

Extend the docstring's numbered list with a fourth entry:

4. **Sense gate** (PHON-205) — a card that depicts a sense the word does not
   carry is dropped before selection and before inheritance. Verdicts come
   from the committed ``data/mappings/image_sense_verdicts.tsv``; only
   ``incompatible`` acts, ``abstain`` never does.

[ ] Step 4: Run test to verify it passes

Run: uv run python -m pytest packages/data/tests/runtime/test_word_images.py -v Expected: PASS — all pre-existing tests plus the 4 new ones.

[ ] Step 5: Plumb it through emit_parquet

In packages/data/src/phonolex_data/runtime/emit_parquet.py, add SENSE_VERDICTS_TSV to the import block from .images (line ~24), add the parameter to emit_parquet after image_blocklist_path:

    image_verdicts_path: Path | None = SENSE_VERDICTS_TSV,

and pass it in the build_word_images call at line ~211:

        verdicts_path=image_verdicts_path,

[ ] Step 6: Run the full data suite

Run: uv run python -m pytest packages/data/tests/ -v --ignore=packages/data/tests/test_datasets.py --ignore=packages/data/tests/test_new_loaders.py Expected: PASS. These are the two files CI excludes.

[ ] Step 7: Commit

git add packages/data/src/phonolex_data/runtime/images.py \
        packages/data/src/phonolex_data/runtime/emit_parquet.py \
        packages/data/tests/runtime/test_word_images.py
git commit -m "feat(phon-205): drop sense-incompatible cards before selection and inheritance"

Task 7: Align every card to a synset (gated, spends money) — AS BUILT¶

Files: - Create: research/2026-07-23-phon-205-sense-gate/align_card_senses.py - Modify: data/mappings/card_senses.tsv

Rewritten from the original plan. The original aligned only the generated + OpenMoji streams and left Mulberry to a deterministic category→lexname bridge. That bridge was deleted during Task 5 (see the refactor note in Task 4 and the build_card_senses.py docstring): Mulberry's symbol_category is a pragmatic AAC taxonomy that does not map cleanly onto WordNet lexnames. All three providers now go through this one alignment pass, reading the hint column build_card_senses.py emits (symbol_grammar + symbol_category for Mulberry/OpenMoji; the cardspec gloss joined in here for generated).

Interfaces: - Consumes: every card_senses.tsv row (all are needs-alignment); the hint column; and cardspec glosses at research/2026-07-12-image-gen-style-probe/bakeoff/cardspec_t*.parquet. - Produces: the same TSV with card_synset filled and source_signal=aligned.

Do not start without explicit user approval — it calls a paid API. ~12.4K distinct calls after dedup (~$5–10).

[ ] Step 1: Pilot and confirm accuracy before the full spend

The script takes --limit N to align only the first N distinct calls and print them without writing the TSV. Run --limit 60, hand-check the sample, and only proceed if accuracy clears ~80% and the errors are in the safe direction — a wrong answer should be NONE (which becomes abstain, dropping nothing), not a confident wrong synset (which could drop a good card).

[ ] Step 2: Key gotchas the script must handle
WordNet is not thread-safe. Its lazy zipfile reader races under the thread pool — pre-warming is not enough because each POS data file opens on first use. Serialize every wn.synsets/.definition() access behind a dedicated lock; the lookups are microseconds so the API calls still overlap.
Dedup by (word, description). The synset a card depicts depends only on the word and what the picture shows, so identical pairs share one answer — ~13.4K rows collapse to ~12.4K calls, and the answer fans back out to every row with that key on write.
Closed output space. Offer only the word's own synsets; validate the returned name against that candidate set before storing it, so a bad answer is a wrong pick, never an invented id.
Checkpoint every 200 answers to a JSON state file and resume from it, per the long-running-jobs policy in CLAUDE.md.
Use the model + client the existing image pipeline uses (gpt-5.4-mini, client.chat.completions.create).
Env: openai is not in the repo venv by default — uv pip install --python .venv/bin/python openai once, then run with .venv/bin/python (the uv-provisioned env lacks WordNet data).
[ ] Step 3: CHECKPOINT — report pilot accuracy, get approval for the full run
[ ] Step 4: Run the full alignment, then rebuild and re-inspect verdicts

set -a; . ./.env; set +a
.venv/bin/python research/2026-07-23-phon-205-sense-gate/align_card_senses.py
.venv/bin/python packages/data/scripts/build_image_sense_gate.py

The generated stream (9,724 cards, 3× Mulberry, model-derived term 1) has an unknown incompatible rate — do not assume the Mulberry prototype's 3.8% carries over. Report the new provider × verdict breakdown.

[ ] Step 5: CHECKPOINT — human review of the real verdicts

This is the plan's core review gate, now with all providers resolved: total incompatible split by tier and provider, how many flagged words have no fallback (go imageless), and a sample for eyeballing. Nothing is committed until reviewed.

[ ] Step 6: Commit

git add data/mappings/card_senses.tsv data/mappings/image_sense_verdicts.tsv \
        research/2026-07-23-phon-205-sense-gate/align_card_senses.py
git commit -m "feat(phon-205): align all card senses via one offline pass"

Task 8: Remediation batch (gated, spends money)¶

Files: - Modify: data/mappings/generated_word_images.tsv - Modify: data/mappings/image_policy_blocklist.tsv

Interfaces: - Consumes: incompatible rows from image_sense_verdicts.tsv (Tasks 5, 7). - Produces: new cards in the generated mapping; sense-divergent undepictables in the blocklist with reason=sense-divergent.

Ship this in the same release as Task 6. 88% of third-party pairs have no fallback provider, so the gate alone strictly removes coverage.

[ ] Step 1: Build the remediation worklist

Create research/2026-07-23-phon-205-sense-gate/build_remediation_worklist.py:

"""Split sense-incompatible words into generate / blocklist (PHON-205).

Depictability is the existing cardspec call -- UNDEPICTABLE or branded goes to
the imageless blocklist, everything else to the generation lane. Words with no
cardspec row need one first; they are reported separately, not silently dropped.
"""

from __future__ import annotations

import json
from pathlib import Path

import polars as pl

REPO_ROOT = Path(__file__).resolve().parents[2]
HERE = Path(__file__).resolve().parent
VERDICTS = REPO_ROOT / "data/mappings/image_sense_verdicts.tsv"
BAKEOFF = REPO_ROOT / "research/2026-07-12-image-gen-style-probe/bakeoff"


def main() -> None:
    bad = (
        pl.read_csv(VERDICTS, separator="\t", schema_overrides={"word": pl.Utf8})
        .filter(pl.col("verdict") == "incompatible")
        .get_column("word")
        .unique()
        .to_list()
    )

    spec: dict[str, dict] = {}
    for path in sorted(BAKEOFF.glob("cardspec_t*.parquet")):
        for r in pl.read_parquet(path).iter_rows(named=True):
            spec[r["word"]] = r

    generate, blocklist, need_spec = [], [], []
    for word in sorted(bad):
        r = spec.get(word)
        if r is None:
            need_spec.append(word)
        elif r.get("branded") or (r.get("depiction") or "").strip().upper() == "UNDEPICTABLE":
            blocklist.append(word)
        else:
            generate.append(word)

    out = {"generate": generate, "blocklist": blocklist, "need_cardspec": need_spec}
    (HERE / "remediation_worklist.json").write_text(json.dumps(out, indent=2))
    print(f"incompatible words : {len(bad):,}")
    print(f"  -> generate      : {len(generate):,}  (~${0.006 * len(generate):.2f})")
    print(f"  -> blocklist     : {len(blocklist):,}")
    print(f"  -> need cardspec : {len(need_spec):,}  (run cardspec.py for these first)")


if __name__ == "__main__":
    main()

Run: uv run --with polars python research/2026-07-23-phon-205-sense-gate/build_remediation_worklist.py

[ ] Step 2: Vision spot-check the flags before spending

The spec requires this: verify the gate's flags are real before paying to regenerate them. Sample 50 words from remediation_worklist.json's generate list, render each card next to the word it currently ships for, and ask a vision model whether the picture fits that word. Cards live at packages/web/frontend/public/images/words/<image_file> for Mulberry/OpenMoji and https://aac.phonolex.com/<word>.webp for generated.

Report the agreement rate. Below ~80% means the gate is over-flagging and needs tuning before any generation — do not proceed on volume alone.

[ ] Step 3: CHECKPOINT — report the split and get approval

Report: how many to generate, how many to blocklist, how many need a cardspec first, the vision agreement rate, estimated cost (~$0.003–0.01/image), and how many words end up imageless either way.

[ ] Step 4: Generate

Use the existing lane at research/2026-07-12-image-gen-style-probe/bakeoff/gen_tranche.py — it is checkpointed and thread-pooled. Then rasterize with rasterize_webp.py.

[ ] Step 5: Review the new cards

Run the PHON-185 rubric review. The independent judge is load-bearing (it caught trade dress and graphic-violence cards in the original program) — do not skip it.

[ ] Step 6: Update the mappings

Add passing cards to generated_word_images.tsv; add undepictable/failed words to image_policy_blocklist.tsv with reason=sense-divergent. A word that cannot be depicted correctly stays imageless — that is the accepted policy, not a failure.

[ ] Step 7: Upload pixels and verify

uv run --with boto3 python packages/web/workers/scripts/upload-generated-images-r2.py
uv run --no-project python packages/web/workers/scripts/verify-generated-images-r2.py

The --no-project flag is required — a plain uv run syncs a repo-root .venv that collides with the MkDocs step and has broken a deploy before.

[ ] Step 8: Full rebuild and end-to-end verification

uv run python packages/data/scripts/build_runtime_parquet.py
uv run python -c "
import polars as pl
wi = pl.read_parquet('data/runtime/word_images.parquet')
print('total cards:', wi.height)
print(wi.filter(pl.col('word').is_in(['rights','right','papers','cats','cat'])))
"

Expected: rights either absent or carrying its own non-inherited card; cats still inherits cat.

[ ] Step 9: Commit

git add data/mappings/generated_word_images.tsv data/mappings/image_policy_blocklist.tsv
git commit -m "feat(phon-205): remediate sense-divergent cards"

Release Notes¶

The seed must be rebuilt and uploaded before this reaches staging — word_images and words.has_image both change:

uv run python packages/web/workers/scripts/export-to-d1.py
uv run --with boto3 python packages/web/workers/scripts/upload-seed-to-r2.py
git add packages/web/workers/scripts/d1-seed.manifest.json

The manifest is the git pointer CI watches; d1-seed.sql itself is gitignored and never committed.