Skip to content

Norms Recompute + Canonical Denylist (v7.0.2) Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Drop residual junk from the canonical set (PHON-197 denylist) and fill the 7 shipped LLM-cloze norms for the ~4K newly-canonical words that carry NULL norms (PHON-196), re-coupling canonical ⊆ normed, in one seed regen.

Architecture: A build-time denylist gate is added to the two-sided is_canonical rule (abbreviation heuristic + proper-dominance signal + curated allow/deny files). After rebuilding words.parquet, the still-un-normed canonical words are extracted to a target list and fed to the existing gpt-4.1-mini cloze build scripts via a new --words-file injection (bypassing their old non-PROPN vocab computation). Norms merge into data/norms/phonolex_*.tsv, then parquet → D1 → R2 seed → deploy.

Tech Stack: Python 3 (phonolex_data pipeline, Polars), pytest, OpenAI API (gpt-4.1-mini), Cloudflare D1 + R2, wrangler.

Global Constraints

  • Norm model is gpt-4.1-mini with the existing prompt templates — same model + prompt as the current column. No other model, no prompt changes. (Verified available 2026-07-21.)
  • 7 in-scope LLM norms: aoa, concreteness, valence, arousal, familiarity, iconicity, boi. socialness excluded. No corpus-derived / CMU-deterministic columns.
  • Norm columns are NULL-tolerant; percentiles are per-norm-population (bisect_right cumulative). Do not assume canonical ⊆ normed anywhere in code.
  • nltk + WordNet are build-time-only dev deps; nothing WordNet-derived ships to D1/client.
  • Denylist re-filters ALL canonical words, not just the gap set (currently-canonical junk like pepsi also drops).
  • d1-seed.sql is a gitignored build product; the committed d1-seed.manifest.json is the git pointer. Reseed is gated on the manifest path. Do NOT trigger deploy-staging via workflow_dispatch (reseeds unconditionally) — trigger via branch push.
  • OpenAI key: OPENAI_API_KEY in repo-root .env (main checkout). The build scripts read it via harness_openai._load_dotenv().
  • Work happens in the worktree .claude/worktrees/norms-7-0-2 on branch fix/norms-recompute-7-0-2. data/norms/*.tsv and data/runtime/*.parquet are gitignored and live in the main checkout's data/ — the worktree shares the repo but not gitignored working files. Run data builds from the main checkout /Users/jneumann/Repos/PhonoLex OR symlink; see Task 0.

Task 0: Worktree data access

Files: - No code changes.

  • [ ] Step 1: Confirm the source data the build needs is reachable from the worktree

The gitignored data/norms/*.tsv, data/cmu/, and data/runtime/*.parquet exist only in the main checkout. Symlink the worktree's data/ to the main checkout's so builds write to one place:

cd /Users/jneumann/Repos/PhonoLex/.claude/worktrees/norms-7-0-2
# data/ tracked files (cmu mappings) are present via git; norms/runtime are gitignored.
# Point the gitignored subdirs at the main checkout so one copy is authoritative.
ls -d data/norms data/runtime 2>/dev/null || true
rm -rf data/norms data/runtime
ln -s /Users/jneumann/Repos/PhonoLex/data/norms data/norms
ln -s /Users/jneumann/Repos/PhonoLex/data/runtime data/runtime
ls -l data/norms/phonolex_aoa.tsv   # resolves via symlink

Expected: phonolex_aoa.tsv resolves. Symlinks are inside gitignored paths, so they won't be committed.

  • [ ] Step 2: Confirm nltk WordNet corpus is present (needed for the canonical rebuild)
cd /Users/jneumann/Repos/PhonoLex
uv run python -c "from nltk.corpus import wordnet as wn; wn.ensure_loaded(); print('wordnet ok', len(list(wn.all_lemma_names())))"

Expected: wordnet ok <N>. If it errors, run uv run python -m nltk.downloader wordnet omw-1.4.

Task 1: Abbreviation + proper-dominance denylist primitives

Files: - Create: packages/data/src/phonolex_data/pipeline/denylist.py - Test: packages/data/tests/test_denylist.py

Interfaces: - Produces: - has_no_vowel(word: str) -> bool - proper_dominance_share(record) -> float — PROPN freq mass ÷ total freq mass (0.0 when no freq). - VOWELS: frozenset[str]

  • [ ] Step 1: Write the failing test
# packages/data/tests/test_denylist.py
from dataclasses import dataclass, field

from phonolex_data.pipeline.denylist import has_no_vowel, proper_dominance_share


@dataclass
class Rec:
    word: str
    all_pos: list = field(default_factory=list)
    all_freqs: list = field(default_factory=list)


def test_has_no_vowel_true_for_consonant_clusters():
    assert has_no_vowel("ws") is True
    assert has_no_vowel("tlc") is True
    assert has_no_vowel("amd") is True


def test_has_no_vowel_false_for_real_words():
    assert has_no_vowel("cat") is False
    assert has_no_vowel("ace") is False
    assert has_no_vowel("apple") is False


def test_has_no_vowel_treats_y_as_vowel():
    # 'y' as the only vowel means it's a real syllable, not an initialism.
    assert has_no_vowel("cry") is False
    assert has_no_vowel("gym") is False


def test_proper_dominance_share():
    # 95% PROPN mass.
    r = Rec("warner", ["PROPN", "NOUN"], [950, 50])
    assert abs(proper_dominance_share(r) - 0.95) < 1e-9


def test_proper_dominance_share_no_freq_is_zero():
    r = Rec("x", [], [])
    assert proper_dominance_share(r) == 0.0


def test_proper_dominance_share_no_propn_is_zero():
    r = Rec("baker", ["NOUN", "VERB"], [800, 200])
    assert proper_dominance_share(r) == 0.0
  • [ ] Step 2: Run test to verify it fails

Run: uv run python -m pytest packages/data/tests/test_denylist.py -v Expected: FAIL — ModuleNotFoundError: phonolex_data.pipeline.denylist

  • [ ] Step 3: Write minimal implementation
# packages/data/src/phonolex_data/pipeline/denylist.py
"""Canonical denylist (PHON-197): drop residual junk the WordNet-common-lemma
realness gate still admits — abbreviations/initialisms, genericized brands, and
proper-dominant homographs. Build-time only; layered on top of the two-sided
is_canonical rule (see pipeline/canonical.py).
"""
from __future__ import annotations

from typing import Any

VOWELS: frozenset[str] = frozenset("aeiouy")


def has_no_vowel(word: str) -> bool:
    """True if the word contains no vowel letter (a,e,i,o,u,y). Catches
    all-consonant initialisms (ws, tlc, amd). 'y' counts as a vowel so real
    y-nucleus words (cry, gym) are NOT flagged."""
    return not any(c in VOWELS for c in word.lower())


def proper_dominance_share(record: Any) -> float:
    """Fraction of a word's corpus frequency mass tagged PROPN, from the full
    all_pos/all_freqs distribution. 0.0 when there is no frequency mass."""
    total = sum(record.all_freqs)
    if total <= 0:
        return 0.0
    propn = sum(
        f for p, f in zip(record.all_pos, record.all_freqs) if p == "PROPN"
    )
    return propn / total
  • [ ] Step 4: Run test to verify it passes

Run: uv run python -m pytest packages/data/tests/test_denylist.py -v Expected: PASS (6 passed)

  • [ ] Step 5: Commit
git add packages/data/src/phonolex_data/pipeline/denylist.py packages/data/tests/test_denylist.py
git commit -m "feat(canonical): denylist primitives — no-vowel + proper-dominance (PHON-197)"

Task 2: Curated denylist files + combined is_denylisted

Files: - Create: data/vocab/canonical-deny.txt - Create: data/vocab/canonical-allow.txt - Modify: packages/data/src/phonolex_data/pipeline/denylist.py - Test: packages/data/tests/test_denylist.py

Interfaces: - Produces: - DenylistConfig (dataclass: deny: frozenset[str], allow: frozenset[str]) - load_denylist_config(vocab_dir: Path | None = None) -> DenylistConfig - is_denylisted(record, cfg: DenylistConfig, propn_threshold: float = 0.90) -> bool - Consumes: has_no_vowel, proper_dominance_share (Task 1).

  • [ ] Step 1: Create the curated files (seed content from PHON-197 examples)

data/vocab/canonical-deny.txt (one lowercased word per line; # comments allowed):

# PHON-197 canonical denylist — words the WordNet-common-lemma gate admits but
# that are junk for a clinical vocabulary. Grouped for review; order irrelevant.
# --- abbreviations / initialisms (vowel-containing; no-vowel ones caught by heuristic) ---
aa
aaa
abc
abcs
abo
dec
res
arb
si
sens
mba
dea
rad
biz
# --- genericized brands / trademarks ---
uzi
vaseline
valium
rohypnol
pepsi
bayer
hoover
lumina
postum
# --- proper-dominant homographs the 0.90 signal misses (curated) ---
hays
bates
sargent

data/vocab/canonical-allow.txt:

# PHON-197 allow-list — real content words that are ALSO common surnames/brands
# and must survive the proper-dominance signal + deny heuristics.
baker
mason
turner
carter
cook
mark
frank
rich
grant
bishop
  • [ ] Step 2: Write the failing test
# append to packages/data/tests/test_denylist.py
from pathlib import Path

from phonolex_data.pipeline.denylist import (
    DenylistConfig,
    load_denylist_config,
    is_denylisted,
)


@dataclass
class FullRec(Rec):
    pass


CFG = DenylistConfig(
    deny=frozenset({"pepsi", "aaa", "hays"}),
    allow=frozenset({"baker", "mason"}),
)


def test_curated_deny_word_dropped():
    r = FullRec("pepsi", ["NOUN"], [900])
    assert is_denylisted(r, CFG) is True


def test_no_vowel_word_dropped_by_heuristic():
    r = FullRec("tlc", ["NOUN"], [900])
    assert is_denylisted(r, CFG) is True


def test_proper_dominant_word_dropped_by_signal():
    # 95% PROPN, not in any curated list -> dropped by the 0.90 signal.
    r = FullRec("warner", ["PROPN", "NOUN"], [950, 50])
    assert is_denylisted(r, CFG) is True


def test_allow_list_protects_occupational_noun():
    # baker is 92% PROPN in corpus but allow-listed -> kept.
    r = FullRec("baker", ["PROPN", "NOUN"], [920, 80])
    assert is_denylisted(r, CFG) is False


def test_ordinary_content_word_kept():
    r = FullRec("mountain", ["NOUN"], [900])
    assert is_denylisted(r, CFG) is False


def test_loanword_kept():
    r = FullRec("gateau", ["NOUN"], [900])
    assert is_denylisted(r, CFG) is False


def test_load_denylist_config_reads_files():
    cfg = load_denylist_config()
    assert "pepsi" in cfg.deny
    assert "baker" in cfg.allow
    assert "# " not in "".join(cfg.deny)  # comments stripped
  • [ ] Step 3: Run test to verify it fails

Run: uv run python -m pytest packages/data/tests/test_denylist.py -v Expected: FAIL — ImportError: cannot import name 'DenylistConfig'

  • [ ] Step 4: Write the implementation (append to denylist.py)
# append to packages/data/src/phonolex_data/pipeline/denylist.py
from dataclasses import dataclass
from pathlib import Path

_VOCAB_DIR = Path(__file__).resolve().parents[4] / "data" / "vocab"


@dataclass(frozen=True)
class DenylistConfig:
    deny: frozenset[str]
    allow: frozenset[str]


def _read_wordlist(path: Path) -> frozenset[str]:
    if not path.exists():
        return frozenset()
    out = set()
    for line in path.read_text(encoding="utf-8").splitlines():
        line = line.strip().lower()
        if not line or line.startswith("#"):
            continue
        out.add(line)
    return frozenset(out)


def load_denylist_config(vocab_dir: Path | None = None) -> DenylistConfig:
    d = vocab_dir or _VOCAB_DIR
    return DenylistConfig(
        deny=_read_wordlist(d / "canonical-deny.txt"),
        allow=_read_wordlist(d / "canonical-allow.txt"),
    )


def is_denylisted(record: Any, cfg: DenylistConfig,
                  propn_threshold: float = 0.90) -> bool:
    """True if the word should be dropped from canonical. Allow-list wins over
    every deny signal. Applied AFTER the two-sided is_canonical gates."""
    word = record.word.lower()
    if word in cfg.allow:
        return False
    if word in cfg.deny:
        return True
    if len(word) <= 4 and has_no_vowel(word):
        return True
    if proper_dominance_share(record) >= propn_threshold:
        return True
    return False

Note the parents[4]: denylist.py is at packages/data/src/phonolex_data/pipeline/denylist.py; parents[4] = repo root. Verify in Step 5's test (test_load_denylist_config_reads_files).

  • [ ] Step 5: Run test to verify it passes

Run: uv run python -m pytest packages/data/tests/test_denylist.py -v Expected: PASS (all). If test_load_denylist_config_reads_files fails on path, adjust parents[N] until _VOCAB_DIR resolves to <repo>/data/vocab.

  • [ ] Step 6: Commit
git add packages/data/src/phonolex_data/pipeline/denylist.py packages/data/tests/test_denylist.py data/vocab/canonical-deny.txt data/vocab/canonical-allow.txt
git commit -m "feat(canonical): curated deny/allow lists + is_denylisted (PHON-197)"

Task 3: Wire the denylist into the canonical rule

Files: - Modify: packages/data/src/phonolex_data/pipeline/canonical.py - Modify: packages/data/src/phonolex_data/pipeline/words.py:262-266 - Test: packages/data/tests/test_canonical.py

Interfaces: - Consumes: is_denylisted, DenylistConfig, load_denylist_config (Task 2). - Produces: is_canonical(record, real_words, thr, denylist_cfg=None) and assign_canonicality(words, real_words, thr, denylist_cfg=None) — new optional last param, backward compatible (existing calls with denylist_cfg=None unchanged).

  • [ ] Step 1: Write the failing test (append to test_canonical.py)
# append to packages/data/tests/test_canonical.py
from phonolex_data.pipeline.denylist import DenylistConfig

DENY = DenylistConfig(deny=frozenset({"pepsi"}), allow=frozenset({"baker"}))


def test_denylisted_brand_not_canonical():
    # Passes both two-sided gates but is on the deny list.
    r = Rec("pepsi", ["NOUN"], [900], frequency=50.0, contextual_diversity=40.0)
    REAL_P = REAL | {"pepsi", "baker", "warner"}
    assert is_canonical(r, real_words=REAL_P, thr=THR, denylist_cfg=DENY) is False


def test_proper_dominant_dropped_but_allowlisted_kept():
    REAL_P = REAL | {"baker", "warner"}
    warner = Rec("warner", ["PROPN", "NOUN"], [950, 50], frequency=50.0,
                 contextual_diversity=40.0)
    baker = Rec("baker", ["PROPN", "NOUN"], [920, 80], frequency=50.0,
                contextual_diversity=40.0)
    assert is_canonical(warner, real_words=REAL_P, thr=THR, denylist_cfg=DENY) is False
    assert is_canonical(baker, real_words=REAL_P, thr=THR, denylist_cfg=DENY) is True


def test_denylist_none_is_backward_compatible():
    r = Rec("pepsi", ["NOUN"], [900], frequency=50.0, contextual_diversity=40.0)
    REAL_P = REAL | {"pepsi"}
    # No denylist -> old behavior, canonical.
    assert is_canonical(r, real_words=REAL_P, thr=THR) is True

Also update the Rec dataclass in test_canonical.py to carry all_pos/all_freqs (already present) — no change needed since is_denylisted reads those. Confirm Rec has no phonemes requirement (it doesn't; denylist uses word + freqs only).

  • [ ] Step 2: Run test to verify it fails

Run: uv run python -m pytest packages/data/tests/test_canonical.py -v Expected: FAIL — is_canonical() got an unexpected keyword argument 'denylist_cfg'

  • [ ] Step 3: Implement — extend canonical.py

Edit is_canonical and assign_canonicality:

# packages/data/src/phonolex_data/pipeline/canonical.py
from phonolex_data.pipeline.denylist import DenylistConfig, is_denylisted


def is_canonical(record: Any, real_words: set[str],
                 thr: CanonicalThresholds,
                 denylist_cfg: "DenylistConfig | None" = None) -> bool:
    if not (
        len(record.word) > 1
        and _content_pos_present(record, thr)
        and _passes_realness(record, real_words, thr)
    ):
        return False
    if denylist_cfg is not None and is_denylisted(record, denylist_cfg):
        return False
    return True


def assign_canonicality(words: dict[str, Any], real_words: set[str],
                        thr: CanonicalThresholds,
                        denylist_cfg: "DenylistConfig | None" = None) -> int:
    n = 0
    for record in words.values():
        record.is_canonical = is_canonical(record, real_words, thr, denylist_cfg)
        if record.is_canonical:
            n += 1
    return n

(Import at top of file; use the string annotation form already shown to avoid any circular-import risk — denylist.py does not import canonical.py, so a direct import is safe too.)

  • [ ] Step 4: Wire into the pipeline — edit words.py:262-266

Replace:

    common_lemmas = load_wordnet_common_lemmas()
    real_words = build_real_word_set(words.keys(), common_lemmas)
    n_canonical = assign_canonicality(words, real_words, PRODUCTION_THRESHOLDS)
    print(f"  canonical: {n_canonical:,} / {len(words):,} words")

with:

    common_lemmas = load_wordnet_common_lemmas()
    real_words = build_real_word_set(words.keys(), common_lemmas)
    denylist_cfg = load_denylist_config()
    n_canonical = assign_canonicality(
        words, real_words, PRODUCTION_THRESHOLDS, denylist_cfg
    )
    print(f"  canonical: {n_canonical:,} / {len(words):,} words "
          f"(denylist: {len(denylist_cfg.deny)} deny / {len(denylist_cfg.allow)} allow)")

Add the import near the existing canonical import (line ~28):

from phonolex_data.pipeline.canonical import assign_canonicality, PRODUCTION_THRESHOLDS
from phonolex_data.pipeline.denylist import load_denylist_config
  • [ ] Step 5: Update the wiring integration test in test_canonical.py

test_build_words_wires_two_sided_rule_and_drops_repair_pass inspects build_words source. Add an assertion that the denylist is wired:

    assert "load_denylist_config" in src
  • [ ] Step 6: Run the full canonical suite

Run: uv run python -m pytest packages/data/tests/test_canonical.py packages/data/tests/test_denylist.py -v Expected: PASS (all).

  • [ ] Step 7: Commit
git add packages/data/src/phonolex_data/pipeline/canonical.py packages/data/src/phonolex_data/pipeline/words.py packages/data/tests/test_canonical.py
git commit -m "feat(canonical): apply denylist gate in is_canonical + pipeline wiring (PHON-197)"

Task 4: Rebuild parquet + canonical ADDED/DROPPED QA

Files: - Create: research/2026-07-21-canonical-denylist/qa.py - Create: research/2026-07-21-canonical-denylist/README.md - Regenerates (gitignored): data/runtime/words.parquet

  • [ ] Step 1: Capture the pre-change canonical set
cd /Users/jneumann/Repos/PhonoLex
uv run python -c "
import polars as pl
df = pl.read_parquet('data/runtime/words.parquet')
c = df.filter(pl.col('is_canonical')==1)['word'].to_list()
open('/private/tmp/claude-501/-Users-jneumann-Repos-PhonoLex/6e9dbb0f-3b74-477c-b3a4-c63dd15b84c3/scratchpad/canon_before.txt','w').write('\n'.join(sorted(c)))
print('before canonical:', len(c))
"

Expected: before canonical: 48314 (± the current build).

  • [ ] Step 2: Rebuild the parquet with the denylist active
cd /Users/jneumann/Repos/PhonoLex
uv run python packages/data/scripts/build_runtime_parquet.py 2>&1 | tail -20

Expected: log line canonical: <N> / <M> words (denylist: 27 deny / 10 allow) with N a few hundred below 48,314.

  • [ ] Step 3: Write the QA diff script
# research/2026-07-21-canonical-denylist/qa.py
"""PHON-197 QA: what the denylist DROPPED from canonical, by class, + a
leave-alone sanity check. Run after rebuilding words.parquet."""
import polars as pl
from pathlib import Path

REPO = Path(__file__).resolve().parents[2]
SCRATCH = "/private/tmp/claude-501/-Users-jneumann-Repos-PhonoLex/6e9dbb0f-3b74-477c-b3a4-c63dd15b84c3/scratchpad"

before = set(Path(f"{SCRATCH}/canon_before.txt").read_text().split("\n")) - {""}
df = pl.read_parquet(REPO / "data/runtime/words.parquet")
after = set(df.filter(pl.col("is_canonical") == 1)["word"].to_list())

dropped = sorted(before - after)
added = sorted(after - before)
print(f"dropped {len(dropped)}  added {len(added)}")
print("\n-- sample dropped (first 60) --")
print(", ".join(dropped[:60]))

# Leave-alone guard: these MUST remain canonical.
must_keep = ["baker", "mason", "turner", "signor", "gateau", "arabian",
             "bostonian", "hello", "carbonara", "pilsner"]
missing = [w for w in must_keep if w in before and w not in after]
print("\n-- leave-alone false-drops (MUST be empty):", missing)
assert not missing, f"FALSE DROPS: {missing}"
print("QA OK")
  • [ ] Step 4: Run the QA script

Run: cd /Users/jneumann/Repos/PhonoLex && uv run python research/2026-07-21-canonical-denylist/qa.py Expected: QA OK, an empty leave-alone list, and a dropped sample dominated by abbreviations/brands/surnames.

  • [ ] Step 5: Review the dropped list; tune the curated files

Inspect the full dropped list. Move any real-word false-drop into data/vocab/canonical-allow.txt; add any junk that leaked through into data/vocab/canonical-deny.txt. Re-run Steps 2 + 4 until the dropped sample is clean and leave-alone is empty. Record the final per-class counts + a spot-check sample in research/2026-07-21-canonical-denylist/README.md.

  • [ ] Step 6: Commit (curated-list tuning + QA artifacts)
cd /Users/jneumann/Repos/PhonoLex/.claude/worktrees/norms-7-0-2
git add research/2026-07-21-canonical-denylist/ data/vocab/canonical-deny.txt data/vocab/canonical-allow.txt
git commit -m "chore(canonical): PHON-197 denylist QA + tuned deny/allow lists"

(Run git from the worktree; data/vocab/* are tracked files shared across worktrees.)

Task 5: Compute the norm gap set

Files: - Create: research/2026-07-21-norms-recompute/gap_words.py - Produces (gitignored scratch): research/2026-07-21-norms-recompute/gap_words.txt

Interfaces: - Produces: gap_words.txt — one lowercased word per line: is_canonical==1 AND NULL across all 7 in-scope norms.

  • [ ] Step 1: Write the gap-set script
# research/2026-07-21-norms-recompute/gap_words.py
"""PHON-196: canonical words still missing every in-scope LLM norm.
Run AFTER the denylist rebuild (Task 4)."""
import polars as pl
from pathlib import Path

REPO = Path(__file__).resolve().parents[2]
NORMS = ["aoa", "concreteness", "valence", "arousal", "familiarity",
         "iconicity", "boi"]

df = pl.read_parquet(REPO / "data/runtime/words.parquet")
cols = [c for c in NORMS if c in df.columns]
canon = df.filter(pl.col("is_canonical") == 1)
null_mask = pl.all_horizontal([pl.col(c).is_null() for c in cols])
gap = canon.filter(null_mask)["word"].sort().to_list()

out = Path(__file__).parent / "gap_words.txt"
out.write_text("\n".join(gap) + "\n")
print(f"norm cols present: {cols}")
print(f"canonical: {canon.height:,}  gap (all-norms-null): {len(gap):,}")
print(f"wrote {out}")

Note: the norm column names in words.parquet must match NORMS. If export/schema aliases differ (e.g. boi vs an aliased name), the [c for c in NORMS if c in df.columns] filter will silently drop them — assert len(cols) == 7 before trusting the gap count, and reconcile against word_properties column names if any are missing.

  • [ ] Step 2: Run it

Run: cd /Users/jneumann/Repos/PhonoLex && uv run python research/2026-07-21-norms-recompute/gap_words.py Expected: canonical: ~48,0xx gap (all-norms-null): ~3,5xx and gap_words.txt written. Sanity: wc -l gap_words.txt matches the printed gap count.

  • [ ] Step 3: Spot-check the gap list
head -30 /Users/jneumann/Repos/PhonoLex/research/2026-07-21-norms-recompute/gap_words.txt
grep -c . /Users/jneumann/Repos/PhonoLex/research/2026-07-21-norms-recompute/gap_words.txt

Expected: real content words (many PROPN-dominant kingdom-class + newly-canonical inflections), NO abbreviations/brands (those were denylisted in Task 4). If junk remains, return to Task 4 Step 5.

  • [ ] Step 4: Commit the script (not the .txt if gitignored)
cd /Users/jneumann/Repos/PhonoLex/.claude/worktrees/norms-7-0-2
git add research/2026-07-21-norms-recompute/gap_words.py
git commit -m "feat(norms): PHON-196 gap-set extractor (canonical ∩ null norms)"

Task 6: --words-file injection into the 7 build scripts

Files: - Create: research/2026-04-30-llm-word-features/_target_vocab.py - Modify (each): research/2026-04-30-llm-word-features/build_{aoa,concreteness,valence,arousal,familiarity,iconicity,boi}.py - Test: research/2026-04-30-llm-word-features/test_target_vocab.py

Interfaces: - Produces: resolve_vocab(cmu: set[str], freq_content: set[str], words_file: str | None, limit: int) -> list[str] - If words_file is None → sorted(cmu & freq_content) (unchanged old behavior). - If words_file given → the file's words intersected with cmu (pronunciation must exist), sorted. Does NOT apply the non-PROPN freq filter — this is the whole point.

  • [ ] Step 1: Write the failing test
# research/2026-04-30-llm-word-features/test_target_vocab.py
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent))
from _target_vocab import resolve_vocab  # noqa: E402


def test_no_words_file_uses_intersection():
    cmu = {"cat", "dog", "kingdom"}
    freq = {"cat", "dog"}  # kingdom filtered out of freq (PROPN)
    assert resolve_vocab(cmu, freq, None, 0) == ["cat", "dog"]


def test_words_file_bypasses_freq_filter(tmp_path):
    # kingdom is PROPN-dominant (not in freq_content) but IS in cmu — must appear.
    wf = tmp_path / "gap.txt"
    wf.write_text("kingdom\nmarch\nzzz_not_in_cmu\n")
    cmu = {"kingdom", "march", "cat"}
    freq = {"cat"}
    assert resolve_vocab(cmu, freq, str(wf), 0) == ["kingdom", "march"]


def test_limit_truncates(tmp_path):
    wf = tmp_path / "gap.txt"
    wf.write_text("a\nb\nc\nd\n")
    cmu = {"a", "b", "c", "d"}
    assert resolve_vocab(cmu, set(), str(wf), 2) == ["a", "b"]
  • [ ] Step 2: Run test to verify it fails

Run: cd /Users/jneumann/Repos/PhonoLex/research/2026-04-30-llm-word-features && uv run python -m pytest test_target_vocab.py -v Expected: FAIL — ModuleNotFoundError: _target_vocab

  • [ ] Step 3: Implement _target_vocab.py
# research/2026-04-30-llm-word-features/_target_vocab.py
"""Shared vocabulary resolver for the norm build scripts (PHON-196).

Default (words_file=None) keeps the original CMU ∩ non-PROPN-freq scope. When a
--words-file is supplied it OVERRIDES that scope with an explicit target list
(intersected only with CMU so a pronunciation exists) — this is how the
newly-canonical PROPN-dominant words get rated, which the non-PROPN freq filter
would otherwise exclude.
"""
from __future__ import annotations

from pathlib import Path


def resolve_vocab(cmu: set[str], freq_content: set[str],
                  words_file: str | None, limit: int) -> list[str]:
    if words_file:
        wanted = {
            w.strip().lower()
            for w in Path(words_file).read_text(encoding="utf-8").splitlines()
            if w.strip() and not w.startswith("#")
        }
        vocab = sorted(wanted & cmu)
    else:
        vocab = sorted(cmu & freq_content)
    if limit:
        vocab = vocab[:limit]
    return vocab
  • [ ] Step 4: Run test to verify it passes

Run: cd /Users/jneumann/Repos/PhonoLex/research/2026-04-30-llm-word-features && uv run python -m pytest test_target_vocab.py -v Expected: PASS (3 passed)

  • [ ] Step 5: Wire resolve_vocab into each of the 7 build scripts

For EACH of build_aoa.py, build_concreteness.py, build_valence.py, build_arousal.py, build_familiarity.py, build_iconicity.py, build_boi.py, make the same two edits.

(a) Add the import near the top (after the existing from harness import ...):

from _target_vocab import resolve_vocab  # type: ignore[import-not-found]

(b) In amain, replace the vocab-building block. Current (identical across scripts):

    cmu = load_cmu_words()
    freq_content = load_content_freq_words()
    common = sorted(cmu & freq_content)
    print(f"[scope] CMU({len(cmu):,}) ∩ non-PROPN freq({len(freq_content):,}) = {len(common):,} words")

    if args.limit:
        common = common[: args.limit]
        print(f"[limit] truncated to {len(common):,}")

Replace with:

    cmu = load_cmu_words()
    freq_content = load_content_freq_words()
    common = resolve_vocab(cmu, freq_content, args.words_file, args.limit)
    scope = f"words-file={args.words_file}" if args.words_file else "CMU ∩ non-PROPN freq"
    print(f"[scope] {scope} -> {len(common):,} words")

(c) Add the argparse arg in main() next to --limit (each script):

    p.add_argument("--words-file", default=None,
                   help="Explicit target word list (one per line); overrides the "
                        "default CMU∩non-PROPN-freq scope, keeping PROPN-dominant words")
  • [ ] Step 6: Smoke-test the injection with a real 3-word kingdom-class run
cd /Users/jneumann/Repos/PhonoLex
printf 'kingdom\nmarch\nfrank\n' > /private/tmp/claude-501/-Users-jneumann-Repos-PhonoLex/6e9dbb0f-3b74-477c-b3a4-c63dd15b84c3/scratchpad/smoke.txt
uv run python research/2026-04-30-llm-word-features/build_aoa.py \
  --words-file /private/tmp/claude-501/-Users-jneumann-Repos-PhonoLex/6e9dbb0f-3b74-477c-b3a4-c63dd15b84c3/scratchpad/smoke.txt \
  --out /private/tmp/claude-501/-Users-jneumann-Repos-PhonoLex/6e9dbb0f-3b74-477c-b3a4-c63dd15b84c3/scratchpad/smoke_aoa.tsv
cat /private/tmp/claude-501/-Users-jneumann-Repos-PhonoLex/6e9dbb0f-3b74-477c-b3a4-c63dd15b84c3/scratchpad/smoke_aoa.tsv

Expected: [scope] words-file=... -> 3 words, then a TSV with kingdom, march, frank and finite aoa values. This proves PROPN-dominant words are now rated (they would be absent under the old filter). ~$0.001 cost.

  • [ ] Step 7: Commit
cd /Users/jneumann/Repos/PhonoLex/.claude/worktrees/norms-7-0-2
git add research/2026-04-30-llm-word-features/_target_vocab.py \
        research/2026-04-30-llm-word-features/test_target_vocab.py \
        research/2026-04-30-llm-word-features/build_aoa.py \
        research/2026-04-30-llm-word-features/build_concreteness.py \
        research/2026-04-30-llm-word-features/build_valence.py \
        research/2026-04-30-llm-word-features/build_arousal.py \
        research/2026-04-30-llm-word-features/build_familiarity.py \
        research/2026-04-30-llm-word-features/build_iconicity.py \
        research/2026-04-30-llm-word-features/build_boi.py
git commit -m "feat(norms): --words-file target injection for the 7 norm build scripts (PHON-196)"

Task 7: Run the 7 norm builds over the gap set

Files: - Appends to (gitignored): data/norms/phonolex_{aoa,concreteness,valence,arousal,familiarity,iconicity,boi}.tsv - Create: research/2026-07-21-norms-recompute/run_all.sh - Create: research/2026-07-21-norms-recompute/validate_gap.py

  • [ ] Step 1: Write the runner (resumable, logged)
# research/2026-07-21-norms-recompute/run_all.sh
#!/usr/bin/env bash
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
GAP=research/2026-07-21-norms-recompute/gap_words.txt
BUILD=research/2026-04-30-llm-word-features
for norm in aoa concreteness valence arousal familiarity iconicity boi; do
  echo "=== [$norm] $(date) ==="
  uv run python "$BUILD/build_${norm}.py" \
    --words-file "$GAP" \
    --out "data/norms/phonolex_${norm}.tsv" \
    --resume \
    --concurrency 6 2>&1 | tee "research/2026-07-21-norms-recompute/${norm}.log"
done
echo "=== all norms done $(date) ==="

--resume skips words already in each TSV, so the existing ~44K rows are untouched and only the gap words are appended. Re-running after an interrupt continues where it left off (append + flush every 50 words is the checkpoint).

  • [ ] Step 2: Run it (long job — ~1–2 h, ~$3–5)

Run: bash research/2026-07-21-norms-recompute/run_all.sh Expected: each norm logs [scope] words-file=... -> ~3,5xx words then progress to 100% with a small fails count. If interrupted, re-run — it resumes.

  • [ ] Step 3: Verify coverage per norm
cd /Users/jneumann/Repos/PhonoLex
GAP=research/2026-07-21-norms-recompute/gap_words.txt
for norm in aoa concreteness valence arousal familiarity iconicity boi; do
  uv run python -c "
import csv,sys
gap=set(l.strip() for l in open('$GAP') if l.strip())
have=set()
for r in csv.DictReader(open('data/norms/phonolex_${norm}.tsv'),delimiter='\t'):
    have.add(r['word'])
miss=gap-have
print(f'${norm}: gap={len(gap)} covered={len(gap & have)} missing={len(miss)}', list(sorted(miss))[:8])
"
done

Expected: missing=0 (or a tiny residue of CMU-absent words) for every norm.

  • [ ] Step 4: Validate distributions (regression sanity)
# research/2026-07-21-norms-recompute/validate_gap.py
"""Sanity: new gap-word norm values must sit within the existing population's
range (no systematic offset / degenerate constant)."""
import csv
from pathlib import Path
from statistics import mean, pstdev

REPO = Path(__file__).resolve().parents[2]
GAP = set(l.strip() for l in (Path(__file__).parent / "gap_words.txt").read_text().splitlines() if l.strip())
VALCOL = {"aoa": "aoa", "concreteness": "concreteness", "valence": "valence",
          "arousal": "arousal", "familiarity": "familiarity",
          "iconicity": "iconicity", "boi": "boi"}

for norm, col in VALCOL.items():
    old, new = [], []
    for r in csv.DictReader(open(REPO / f"data/norms/phonolex_{norm}.tsv"), delimiter="\t"):
        try:
            v = float(r[col])
        except (ValueError, KeyError):
            continue
        if v != v:  # NaN
            continue
        (new if r["word"] in GAP else old).append(v)
    print(f"{norm:14s} old n={len(old):6d} mean={mean(old):.2f}±{pstdev(old):.2f}  "
          f"new n={len(new):5d} mean={mean(new):.2f}±{pstdev(new):.2f}")

Run: cd /Users/jneumann/Repos/PhonoLex && uv run python research/2026-07-21-norms-recompute/validate_gap.py Expected: new-word means within ~1 rating point of old means, non-degenerate stdev. A wildly off mean (e.g. all ~5.0) signals a prompt/model regression — stop and investigate before reseeding.

  • [ ] Step 5: Record cost + commit artifacts

Sum the API usage (from logs / OpenAI dashboard) into research/2026-07-21-norms-recompute/README.md with per-norm coverage + the validate_gap output.

cd /Users/jneumann/Repos/PhonoLex/.claude/worktrees/norms-7-0-2
git add research/2026-07-21-norms-recompute/run_all.sh \
        research/2026-07-21-norms-recompute/validate_gap.py \
        research/2026-07-21-norms-recompute/README.md
git commit -m "chore(norms): PHON-196 gap-norm run scripts + coverage/validation record"

(The data/norms/*.tsv are gitignored — not committed; they feed the seed.)

Task 8: Rebuild, reseed, version bump, deploy

Files: - Regenerates (gitignored): data/runtime/*.parquet, d1-seed.sql - Modify: packages/web/workers/scripts/d1-seed.manifest.json (git pointer) - Modify: packages/web/frontend/package.json (version → 7.0.2) + version chip/drawer per the version-bump checklist - Modify: CHANGELOG / release notes if present

  • [ ] Step 1: Rebuild parquet from the updated norms + export the seed
cd /Users/jneumann/Repos/PhonoLex
uv run python packages/data/scripts/build_runtime_parquet.py 2>&1 | tail -15
uv run python packages/web/workers/scripts/export-to-d1.py 2>&1 | tail -10

Expected: parquet rebuild logs the canonical count (denylisted, stable vs Task 4) and the norm merges now cover the gap words; d1-seed.sql regenerates.

  • [ ] Step 2: Apply to local D1 and verify gap-word norms resolve
cd /Users/jneumann/Repos/PhonoLex
uv run python packages/web/workers/scripts/chunk-seed-sql.py
cd packages/web/workers
for f in scripts/d1-chunks/chunk_*.sql; do npx wrangler d1 execute phonolex --local --file "$f" >/dev/null; done
# pick a known gap word (from gap_words.txt, e.g. a kingdom-class word) and confirm norms are non-null
npx wrangler d1 execute phonolex --local --command \
  "SELECT w.word, wp.aoa, wp.concreteness, wp.iconicity, wp.boi FROM words w JOIN word_properties wp ON wp.word=w.word WHERE w.word='kingdom';"

Expected: kingdom (or your chosen gap word) returns non-NULL aoa/concreteness/iconicity/boi. Also confirm a denylisted word is now non-canonical: SELECT word,is_canonical FROM words WHERE word='pepsi'; → 0.

  • [ ] Step 3: Run the worker test suite against the reseeded local D1
cd /Users/jneumann/Repos/PhonoLex/packages/web/workers && npm test 2>&1 | tail -20

Expected: PASS. (Catches any schema/shape regression from the reseed.)

  • [ ] Step 4: Upload the seed to R2 + update the manifest pointer
cd /Users/jneumann/Repos/PhonoLex
uv run --with boto3 python packages/web/workers/scripts/upload-seed-to-r2.py 2>&1 | tail -10
git -C .claude/worktrees/norms-7-0-2 status --short packages/web/workers/scripts/d1-seed.manifest.json

Expected: upload succeeds (or skips if hash already present); d1-seed.manifest.json shows a new object key + sha256.

  • [ ] Step 5: Version bump to 7.0.2

Follow the version-bump checklist (AppHeader chip + any release drawer). Edit packages/web/frontend/package.json "version": "7.0.2" and the in-app version reference(s). (Coordinate the exact number with the 7.0.1 mobile branch at merge time — if 7.0.1 lands first, 7.0.2 is correct; otherwise adjust.)

  • [ ] Step 6: Commit + push the branch
cd /Users/jneumann/Repos/PhonoLex/.claude/worktrees/norms-7-0-2
git add packages/web/workers/scripts/d1-seed.manifest.json packages/web/frontend/package.json
# + any version-chip files touched
git commit -m "chore(release): v7.0.2 — canonical denylist + norm recompute reseed (PHON-196/197)"
git push
  • [ ] Step 7: Open PR → develop; let CI reseed staging

Open a PR from fix/norms-recompute-7-0-2 → develop. The manifest change triggers the staging reseed on merge (do NOT workflow_dispatch). After staging deploys, verify a gap word on staging:

curl -s "https://staging-api.phonolex.com/api/words/kingdom" | python3 -c "import sys,json;d=json.load(sys.stdin);print({k:d.get(k) for k in ('word','is_canonical','aoa','concreteness','iconicity','boi')})"

Expected: is_canonical: 1 and non-null norms. Spot-check a denylisted word (pepsi) resolves but with is_canonical: 0 and is absent from Word Lists candidate results.

  • [ ] Step 8: Promote to prod after review

Open develop → main PR. Merging applies the seed to prod D1. Verify the same curl against api.phonolex.com. Then clean up the worktree (git worktree remove), and move PHON-196 + PHON-197 to Done.

Self-Review Notes

  • Spec coverage: Part A (denylist) → Tasks 1–4; Part B (recompute) → Tasks 5–7; rebuild/reseed/deploy → Task 8. All 7 norms enumerated in the Global Constraints and Tasks 5/6/7. Acceptance criteria map: canonical junk reduced (Task 4 QA), gap→0 (Task 7 Step 3), homogeneity (Global Constraint + Task 7 Step 4), percentiles recomputed (Task 8 Step 1, automatic in the pipeline), cost recorded (Task 7 Step 5).
  • The load-bearing subtlety (spec Risks): the build scripts' internal non-PROPN filter is bypassed only via --words-file (Task 6) — the smoke test in Task 6 Step 6 is the guard that PROPN-dominant words actually get rated.
  • Backward compatibility: denylist_cfg defaults to None in is_canonical/assign_canonicality, so existing test_canonical.py calls are unaffected (Task 3 Step 1 test test_denylist_none_is_backward_compatible).