PHON-109 — Productionization Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Move CSP + reranker_v2 stack from spike to phonolex_generators.csp. Replace v6/T5Gemma server path with three new routes. Bundle 3 follow-ups during the move. Retire RunPod.

Architecture: spike → packages/generators/src/phonolex_generators/csp/. Server cold-starts CSP + reranker. Worker drops RunPod logic; will point at new container endpoint (deployment is sibling ticket).

Tech Stack: Polars, LightGBM, sentence-transformers (existing). FastAPI server (existing structure). Hono Worker (existing).

---

File map

Files moved (spike → package)

Spike path	Package path
<spike>/pair_driven.py	packages/generators/src/phonolex_generators/csp/pair_driven.py
<spike>/verb_candidates.py	packages/generators/src/phonolex_generators/csp/verb_candidates.py
<spike>/skeleton_csp.py	packages/generators/src/phonolex_generators/csp/skeleton.py
<spike>/constraint_surface.py	packages/generators/src/phonolex_generators/csp/constraints.py
<spike>/paragraph_csp.py	packages/generators/src/phonolex_generators/csp/paragraph.py
<spike>/quality_axis_v2.py	packages/generators/src/phonolex_generators/csp/reranker/predict.py
<spike>/train_reranker_v2.py	packages/generators/src/phonolex_generators/csp/reranker/train.py
<spike>/reranker_v2.py	packages/generators/src/phonolex_generators/csp/reranker/rerank.py
<spike>/active_learning_select.py	packages/generators/src/phonolex_generators/csp/reranker/active_learning.py
<spike>/embedding_cache.py	packages/generators/src/phonolex_generators/csp/reranker/embedding_cache.py
<spike>/llm_judge.py	packages/generators/src/phonolex_generators/csp/reranker/judge.py
<spike>/outputs/skeletons.parquet	data/runtime/skeletons.parquet
<spike>/outputs/reranker_v2.pkl	data/runtime/reranker_v2.pkl

<spike> = packages/generation/research/2026-05-07-sentence-generation-paradigms/. Spike directory becomes archive (kept in git history; no deletion).

Files modified (server + worker)

• packages/generation/server/main.py — gut T5Gemma startup, add CSP cold-start
• packages/generation/server/schemas.py — new request/response models
• packages/generation/server/routes/generate.py — three new routes
• packages/web/workers/src/routes/generation.ts — drop RunPod, route to new server

Files retired

• packages/generation/server/governor.py (v6 governor)
• packages/generation/server/model.py (T5Gemma loader)
• packages/generation/rp_handler.py (RunPod handler)
• packages/generation/Dockerfile (RunPod image; new Dockerfile for Containers comes in PHON-109b)
• packages/web/workers/src/lib/serverStatus.ts RunPod-specific helpers (will get a Containers-status equivalent in 109b)

Tests: packages/generators/tests/csp/, packages/generation/server/tests/, packages/web/workers/src/__tests__/.

---

Task 1: Move LFS artifacts to data/runtime/

Files: - Move: <spike>/outputs/skeletons.parquet → data/runtime/skeletons.parquet - Move: <spike>/outputs/reranker_v2.pkl → data/runtime/reranker_v2.pkl - Modify: .gitattributes

• [ ] Step 1.1: Verify LFS rule covers the new files

grep "data/runtime" .gitattributes

Expected: data/runtime/*.parquet filter=lfs diff=lfs merge=lfs -text already covers parquet. Need to add *.pkl:

data/runtime/*.pkl filter=lfs diff=lfs merge=lfs -text

• [ ] Step 1.2: Move artifacts

git mv packages/generation/research/2026-05-07-sentence-generation-paradigms/outputs/skeletons.parquet data/runtime/skeletons.parquet
git mv packages/generation/research/2026-05-07-sentence-generation-paradigms/outputs/reranker_v2.pkl data/runtime/reranker_v2.pkl

If reranker_v2.pkl is gitignored (not tracked), move via shell:

mv packages/generation/research/2026-05-07-sentence-generation-paradigms/outputs/reranker_v2.pkl data/runtime/reranker_v2.pkl

• [ ] Step 1.3: Verify LFS objects resolve

ls -la data/runtime/skeletons.parquet data/runtime/reranker_v2.pkl
uv run python -c "
import polars as pl, pickle
print('skeletons rows:', pl.read_parquet('data/runtime/skeletons.parquet').height)
m = pickle.load(open('data/runtime/reranker_v2.pkl', 'rb'))
print('reranker version:', m.get('version'), 'axes:', m.get('axes'))
"

Expected: skeletons ~219K rows; reranker version 2 with 4 axes.

• [ ] Step 1.4: Commit

git add .gitattributes data/runtime/skeletons.parquet data/runtime/reranker_v2.pkl
git commit -m "$(cat <<'EOF'
PHON-109: move skeletons.parquet + reranker_v2.pkl to data/runtime/ (LFS)

Both artifacts move from spike/outputs/ to the canonical runtime
directory. .gitattributes extended to cover .pkl LFS-tracking.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
EOF
)"

---

Task 2: Create phonolex_generators.csp package skeleton

Files: - Create: packages/generators/src/phonolex_generators/csp/__init__.py - Create: packages/generators/src/phonolex_generators/csp/reranker/__init__.py - Create: packages/generators/tests/csp/__init__.py - Create: packages/generators/tests/csp/conftest.py

• [ ] Step 2.1: Create directories + init files

mkdir -p packages/generators/src/phonolex_generators/csp/reranker
mkdir -p packages/generators/tests/csp
touch packages/generators/src/phonolex_generators/csp/__init__.py
touch packages/generators/src/phonolex_generators/csp/reranker/__init__.py
touch packages/generators/tests/csp/__init__.py

• [ ] Step 2.2: Add module docstrings

Edit packages/generators/src/phonolex_generators/csp/__init__.py:

"""PHON-109: pair-driven CSP sentence + paragraph generation.

Productionized from the 2026-05-07 sentence-generation paradigms spike.
The constraint-driven resolver: constraints shape lexicon → pair frame
(if contrastive) → selectional self-join → skeleton host filter →
realize → reranker.
"""

Edit packages/generators/src/phonolex_generators/csp/reranker/__init__.py:

"""PHON-107: per-axis quality reranker.

4 LightGBM models scoring naturalness, grammaticality, age_appropriate,
coherence. Active-learning loop bootstraps from teacher (Sonnet 4.6)
labels.
"""

• [ ] Step 2.3: Conftest with shared fixtures

Create packages/generators/tests/csp/conftest.py:

"""Shared fixtures for CSP tests."""
from pathlib import Path

import polars as pl
import pytest
from phonolex_data.runtime.store import WordStore


REPO_ROOT = Path(__file__).resolve().parents[4]
DATA_RUNTIME = REPO_ROOT / "data" / "runtime"


@pytest.fixture(scope="session")
def store():
    return WordStore.from_parquet(DATA_RUNTIME / "words.parquet")


@pytest.fixture(scope="session")
def sel_df():
    return pl.read_parquet(DATA_RUNTIME / "selectional.parquet")


@pytest.fixture(scope="session")
def skeletons_df():
    return pl.read_parquet(DATA_RUNTIME / "skeletons.parquet")

• [ ] Step 2.4: Verify package imports

cd /Users/jneumann/Repos/PhonoLex && \
  uv run python -c "from phonolex_generators import csp; from phonolex_generators.csp import reranker; print('OK')"

Expected: prints OK.

If phonolex_generators doesn't auto-discover submodules, may need to add from . import csp in phonolex_generators/__init__.py — verify and adjust.

• [ ] Step 2.5: Commit

git add packages/generators/src/phonolex_generators/csp/ \
        packages/generators/tests/csp/
git commit -m "$(cat <<'EOF'
PHON-109: phonolex_generators.csp package skeleton

Empty submodule with __init__ docstrings and shared test fixtures
(store, sel_df, skeletons_df) loading from data/runtime/.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
EOF
)"

---

Task 3: Move + fix pair_driven.py (bundles follow-ups #1 and #2)

Files: - Create: packages/generators/src/phonolex_generators/csp/pair_driven.py - Create: packages/generators/tests/csp/test_pair_driven.py

This task bundles the two follow-ups so the productionized module ships clean: - Follow-up #1: solve() enforces locked_slots for filler roles (currently only V). - Follow-up #2: solve() dedups (verb, fillers, skeleton) tuples before top_k.

• [ ] Step 3.1: Copy spike module

Copy <spike>/pair_driven.py to packages/generators/src/phonolex_generators/csp/pair_driven.py. Update imports: - from constraint_surface import ... → from .constraints import ... - from skeleton_csp import _load_pairs_for_request → from .skeleton import _load_pairs_for_request - from verb_candidates import compute_verb_candidates → from .verb_candidates import compute_verb_candidates

• [ ] Step 3.2: Apply follow-up #1 — locked_slots for filler roles

In solve(), after the contrastive vs non-contrastive branch produces the joined frame and BEFORE skeleton host filter, add:

    # Follow-up: enforce locked_slots for filler roles.
    # The contrastive/non-contrastive branches honor V (via verb_candidates),
    # but filler roles (nsubj, dobj, iobj, pobj_X) are silently ignored.
    # Apply explicit row filter here so callers (paragraph_csp) can lock nsubj.
    for slot, locked_value in locked_slots.items():
        if slot == "V":
            continue  # already enforced earlier
        joined = joined.filter(
            ((pl.col("role_a") == slot) & (pl.col("filler_a") == locked_value))
            | ((pl.col("role_b") == slot) & (pl.col("filler_b") == locked_value))
        )
        if joined.height == 0:
            return []

• [ ] Step 3.3: Apply follow-up #2 — top-K dedup

In solve(), before the candidate-assembly loop's if len(candidates) >= top_k: break, add a dedup set:

    # Follow-up: dedup (verb, fillers, skeleton) tuples — same surface from
    # different pair orientations should appear at most once.
    seen_keys: set[tuple] = set()
    candidates: list[dict] = []
    for row in joined.head(top_k * 4).iter_rows(named=True):  # over-fetch for dedup
        # ... existing skeleton lookup + render logic ...
        key = (
            row["verb"],
            row["role_a"], row["filler_a"],
            row["role_b"], row["filler_b"],
            skel_arg,
        )
        if key in seen_keys:
            continue
        seen_keys.add(key)
        # ... existing candidate dict construction + append ...
        if len(candidates) >= top_k:
            break

Increase the over-fetch multiplier from *2 to *4 to account for dedup losses.

• [ ] Step 3.4: Write tests for the moved module + the two fixes

Create packages/generators/tests/csp/test_pair_driven.py reusing the spike's test_pair_driven_solve.py content but with package imports. Add NEW tests for the follow-ups:

def test_solve_respects_nsubj_lock_with_minpair(store, sel_df, skeletons_df):
    """Follow-up #1: locked_slots[nsubj]=word is enforced in solve()."""
    from phonolex_generators.csp import pair_driven
    from phonolex_generators.csp.constraints import MinpairConstraint

    spec_words = frozenset({"seed", "seas", "drill", "drum"})
    candidates = pair_driven.solve(
        spec_words=spec_words,
        word_df=store.df,
        sel_df=sel_df,
        pairs_df=store.pairs_df,
        skeletons_df=skeletons_df,
        band="fineweb_adult",
        constraints=[MinpairConstraint(phoneme1="d", phoneme2="z", position="final")],
        locked_slots={"nsubj": "seed"},
        top_k=4,
    )
    for c in candidates:
        # nsubj must equal "seed" — the lock must be honored
        nsubj = c["fillers"].get("nsubj")
        assert nsubj == "seed", f"nsubj lock violated: got {nsubj}"


def test_solve_dedups_top_k(store, sel_df, skeletons_df):
    """Follow-up #2: same (verb, fillers, skeleton) appears at most once."""
    from phonolex_generators.csp import pair_driven
    from phonolex_generators.csp.constraints import MinpairConstraint

    candidates = pair_driven.solve(
        spec_words=frozenset({"seed", "seas", "drill", "drum"}),
        word_df=store.df,
        sel_df=sel_df,
        pairs_df=store.pairs_df,
        skeletons_df=skeletons_df,
        band="fineweb_adult",
        constraints=[MinpairConstraint(phoneme1="d", phoneme2="z", position="final")],
        top_k=8,
    )
    keys = set()
    for c in candidates:
        key = (c["verb"], frozenset(c["fillers"].items()), c["skeleton"])
        assert key not in keys, f"dedup violated: {key} appears twice"
        keys.add(key)

The existing tests from spike (test_resolve_contrastive_join_*, test_resolve_non_contrastive_*, test_select_host_skeletons_*, test_solve_minpair_end_to_end, test_solve_exclude_filters_verb, test_solve_produces_renderable_sentences, test_words_with_phoneme_at_position_*, test_resolve_multopp_*) get copied with imports updated.

• [ ] Step 3.5: Run tests, verify pass

cd /Users/jneumann/Repos/PhonoLex && \
  uv run python -m pytest packages/generators/tests/csp/test_pair_driven.py -v

Expected: all pass (existing spike tests + 2 new follow-up tests). The dedup test in particular verifies the behavior change.

• [ ] Step 3.6: Commit

git add packages/generators/src/phonolex_generators/csp/pair_driven.py \
        packages/generators/tests/csp/test_pair_driven.py
git commit -m "$(cat <<'EOF'
PHON-109: move pair_driven.py to phonolex_generators.csp + 2 follow-ups

Follow-up #1: locked_slots[filler_role] is now enforced in solve().
Filters joined frame by (role_a==slot, filler_a==value) | symmetric
for role_b. paragraph_csp's local workarounds become removable in
Task 6.

Follow-up #2: top-K dedup on (verb, role assignments, skeleton) tuple.
Same surface from different pair orientations no longer leaks to top_k.
Over-fetch multiplier raised to 4× to compensate.

Spike pair_driven.py left in place until Task 14 (archive cleanup).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
EOF
)"

---

Task 4: Move verb_candidates.py

Files: - Create: packages/generators/src/phonolex_generators/csp/verb_candidates.py - Create: packages/generators/tests/csp/test_verb_candidates.py

• [ ] Step 4.1: Copy spike module

Copy <spike>/verb_candidates.py to packages/generators/src/phonolex_generators/csp/verb_candidates.py. No internal imports to update (it imports only polars).

• [ ] Step 4.2: Copy + update tests

Copy <spike>/test_verb_candidates.py to packages/generators/tests/csp/test_verb_candidates.py. Replace import verb_candidates with from phonolex_generators.csp import verb_candidates. Drop the spike's sys.path.insert line; package imports work directly.

• [ ] Step 4.3: Run tests

cd /Users/jneumann/Repos/PhonoLex && \
  uv run python -m pytest packages/generators/tests/csp/test_verb_candidates.py -v

Expected: 3 passed (matching spike).

• [ ] Step 4.4: Commit

git add packages/generators/src/phonolex_generators/csp/verb_candidates.py \
        packages/generators/tests/csp/test_verb_candidates.py
git commit -m "PHON-109: move verb_candidates.py to phonolex_generators.csp"

---

Task 5: Move skeleton + constraints + realize modules

Files: - Create: packages/generators/src/phonolex_generators/csp/constraints.py - Create: packages/generators/src/phonolex_generators/csp/skeleton.py - Create: packages/generators/tests/csp/test_skeleton.py - Create: packages/generators/tests/csp/test_constraints.py

• [ ] Step 5.1: Move constraint_surface.py

Copy <spike>/constraint_surface.py to packages/generators/src/phonolex_generators/csp/constraints.py. No internal imports to update.

• [ ] Step 5.2: Move skeleton_csp.py

Copy <spike>/skeleton_csp.py to packages/generators/src/phonolex_generators/csp/skeleton.py. Update imports: - from constraint_surface import ... → from .constraints import ...

• [ ] Step 5.3: Write tests for both modules

Create packages/generators/tests/csp/test_constraints.py:

"""PHON-109: constraint dataclass tests."""
from phonolex_generators.csp.constraints import (
    ExcludeConstraint, IncludeConstraint, BoundConstraint, BoundBoostConstraint,
    MinpairConstraint, MaxoppConstraint, MultoppConstraint,
)


def test_minpair_constraint_construction():
    c = MinpairConstraint(phoneme1="k", phoneme2="b", position="initial")
    assert c.phoneme1 == "k"
    assert c.position == "initial"
    assert c.slots is None
    assert c.type == "contrastive_minpair"


def test_maxopp_constraint_default_min_sonorant_diff():
    c = MaxoppConstraint(phoneme1="k", phoneme2="m")
    assert c.min_sonorant_diff == 0.5


def test_multopp_constraint_n_targets_default():
    c = MultoppConstraint(substitute="t", targets=("s", "ʃ", "tʃ"))
    assert c.n_targets == 3


def test_constraints_are_frozen():
    import dataclasses
    c = ExcludeConstraint(phonemes=("ɹ",))
    with __import__("pytest").raises(dataclasses.FrozenInstanceError):
        c.phonemes = ("k",)

Create packages/generators/tests/csp/test_skeleton.py reusing spike's helper-only tests:

"""PHON-109: skeleton helpers (parse_arg_structure, _load_pairs_for_request)."""
import pytest
from phonolex_generators.csp import skeleton
from phonolex_generators.csp.constraints import MinpairConstraint, MaxoppConstraint


def test_parse_arg_structure():
    assert skeleton.parse_arg_structure("nsubj,V,dobj") == ("nsubj", "V", "dobj")


def test_load_pairs_for_request_basic(store):
    pair_df = skeleton._load_pairs_for_request(
        constraint=MinpairConstraint(phoneme1="k", phoneme2="b", position="initial"),
        pairs_df=store.pairs_df,
        filtered_spec=frozenset(["cat", "bat", "kid", "bid", "key", "bee"]),
    )
    assert pair_df.height > 0
    assert {"filler_a", "filler_b", "feature_distance", "sonorant_diff"} <= set(pair_df.columns)

• [ ] Step 5.4: Run tests

cd /Users/jneumann/Repos/PhonoLex && \
  uv run python -m pytest packages/generators/tests/csp/test_constraints.py packages/generators/tests/csp/test_skeleton.py -v

Expected: ~6 passed.

• [ ] Step 5.5: Commit

git add packages/generators/src/phonolex_generators/csp/constraints.py \
        packages/generators/src/phonolex_generators/csp/skeleton.py \
        packages/generators/tests/csp/test_constraints.py \
        packages/generators/tests/csp/test_skeleton.py
git commit -m "PHON-109: move constraints + skeleton helpers to phonolex_generators.csp"

---

Task 6: Move paragraph_csp.py + remove filler-lock workarounds

Files: - Create: packages/generators/src/phonolex_generators/csp/paragraph.py - Create: packages/generators/tests/csp/test_paragraph.py

The 2 paragraph_csp workarounds for pair_driven.solve's missing filler lock (Task 7 post-filter, lock-after-pick _solve_with_locked_nsubj) are removable now that Task 3 fixed the lock at the source.

• [ ] Step 6.1: Copy spike module

Copy <spike>/paragraph_csp.py to packages/generators/src/phonolex_generators/csp/paragraph.py. Update imports: - import pair_driven → from . import pair_driven - from skeleton_csp import realize, ... → from .skeleton import realize, ... - from verb_candidates import compute_verb_candidates → from .verb_candidates import compute_verb_candidates - from constraint_surface import ... → from .constraints import ...

• [ ] Step 6.2: Remove workaround #1 — Task 7 post-filter

Find the post-filter that drops candidates whose nsubj != discourse_subject (Task 7 fix). It looks like:

# Post-filter to actual nsubj==subject (Task 7 workaround for pair_driven.solve filler-lock gap)
cands = [
    c for c in cands
    if (c.get("role_a") == "nsubj" and c.get("filler_a") == subject)
    or (c.get("role_b") == "nsubj" and c.get("filler_b") == subject)
]

Delete it. With Task 3's fix, pair_driven.solve(locked_slots={"nsubj": subject}) now enforces this server-side.

• [ ] Step 6.3: Remove workaround #2 — _solve_with_locked_nsubj

Find _solve_with_locked_nsubj (the targeted-selectional-query helper added during the lock-after-pick fix) and delete it. Its callers should switch to pair_driven.solve(locked_slots={"nsubj": subject, ...}) directly.

If callers were passing additional kwargs (band, sel_df, etc.) the same kwargs go to pair_driven.solve — verify the signature parity and adapt.

• [ ] Step 6.4: Write paragraph tests

Copy spike's test_paragraph_csp.py to packages/generators/tests/csp/test_paragraph.py. Update imports: - import paragraph_csp → from phonolex_generators.csp import paragraph as paragraph_csp (alias preserves test code) - Or rename the module reference throughout to paragraph

• [ ] Step 6.5: Run tests

cd /Users/jneumann/Repos/PhonoLex && \
  uv run python -m pytest packages/generators/tests/csp/test_paragraph.py -v

Expected: 5+ passed (matching spike's count). The minpair-paragraph and multopp-paragraph tests should still produce real paragraphs (verifying the workaround removal didn't regress).

• [ ] Step 6.6: Commit

git add packages/generators/src/phonolex_generators/csp/paragraph.py \
        packages/generators/tests/csp/test_paragraph.py
git commit -m "$(cat <<'EOF'
PHON-109: move paragraph.py + remove 2 filler-lock workarounds

paragraph.py removes the Task 7 post-filter (nsubj==subject) and the
_solve_with_locked_nsubj helper. Both worked around pair_driven.solve's
missing filler-role lock enforcement; Task 3's fix to pair_driven.solve
makes them unnecessary.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
EOF
)"

---

Task 7: Move reranker submodule

Files: - Create: packages/generators/src/phonolex_generators/csp/reranker/{predict,train,rerank,active_learning,embedding_cache,judge}.py - Create: packages/generators/tests/csp/test_reranker.py

Six modules total. Move them in one task because they share import patterns.

• [ ] Step 7.1: Move modules

From	To
<spike>/quality_axis_v2.py	csp/reranker/predict.py
<spike>/train_reranker_v2.py	csp/reranker/train.py
<spike>/reranker_v2.py	csp/reranker/rerank.py
<spike>/active_learning_select.py	csp/reranker/active_learning.py
<spike>/embedding_cache.py	csp/reranker/embedding_cache.py
<spike>/llm_judge.py	csp/reranker/judge.py

For each, update imports: - from quality_axis_v2 import ... → from .predict import ... - from train_reranker_v2 import ... → from .train import ... - from reranker_v2 import ... → from .rerank import ... - from active_learning_select import ... → from .active_learning import ... - from embedding_cache import ... → from .embedding_cache import ... - v1 helper imports (from train_reranker import _extract_tabular, ...) — these need a decision: - (a) Inline the v1 helpers into csp/reranker/train.py (~50 lines). - (b) Add the v1 spike file to package as csp/reranker/_v1_helpers.py (preserves history).

Choose (a) — inline. Keep the helpers documented as "from PHON-66 v1 reranker; reused for tabular feature parity."

• [ ] Step 7.2: Update DEFAULT_MODEL_PATH

In predict.py:

DEFAULT_MODEL_PATH = Path(__file__).resolve().parents[5] / "data" / "runtime" / "reranker_v2.pkl"

Verify the parents count: predict.py → reranker → csp → phonolex_generators → src → packages → generators → ... actually let me count: - packages/generators/src/phonolex_generators/csp/reranker/predict.py - parents[0] = reranker - parents[1] = csp - parents[2] = phonolex_generators - parents[3] = src - parents[4] = generators - parents[5] = packages - parents[6] = repo root

So parents[6] / "data" / "runtime" / "reranker_v2.pkl".

Verify by running once after the move.

• [ ] Step 7.3: Update train.py's judged.jsonl path

In train.py's load_data:

JUDGED_PATH = Path(__file__).resolve().parents[6] / "packages" / "generation" / "research" / "2026-05-07-sentence-generation-paradigms" / "outputs" / "judged.jsonl"

Or move judged.jsonl too (LFS-track or commit as plain text — it's ~88+200×3 = ~688 entries × ~1KB each = ~700KB). For now, keep the path pointing at the spike's outputs so retraining still works without committing the labels.

• [ ] Step 7.4: Move tests

Copy <spike>/test_reranker_v2.py to packages/generators/tests/csp/test_reranker.py. Update imports per the same mapping. Drop spike's sys.path.insert line.

• [ ] Step 7.5: Run tests

cd /Users/jneumann/Repos/PhonoLex && \
  uv run python -m pytest packages/generators/tests/csp/test_reranker.py -v

Expected: 9 passed (matching spike).

• [ ] Step 7.6: Commit

git add packages/generators/src/phonolex_generators/csp/reranker/ \
        packages/generators/tests/csp/test_reranker.py
git commit -m "$(cat <<'EOF'
PHON-109: move reranker submodule to phonolex_generators.csp.reranker

Six modules: predict, train, rerank, active_learning, embedding_cache,
judge. v1 reranker helpers (_extract_tabular, _request_text,
split_by_group, TABULAR_FEATURES, EMBEDDING_DIM) inlined into train.py
so v1 spike file becomes truly archival.

DEFAULT_MODEL_PATH now resolves to data/runtime/reranker_v2.pkl.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
EOF
)"

---

Task 8: Server cold-start rewrite

Files: - Modify: packages/generation/server/main.py - Modify: packages/generation/server/schemas.py

Replace T5Gemma loading with CSP cold-start. New schemas for the three endpoints.

• [ ] Step 8.1: Inspect current main.py

cat packages/generation/server/main.py

Note any FastAPI middleware, CORS config, lifespan handlers, etc. that should be preserved.

• [ ] Step 8.2: Rewrite main.py

Replace the body with CSP cold-start:

"""PHON-109: FastAPI server hosting CSP + reranker_v2.

Cold-starts WordStore + selectional + skeletons + reranker model on
startup. Per-request: pair_driven.solve → reranker.rerank_with_axes →
return top_k.

Replaces the v6/T5Gemma path retired in PHON-109.
"""
from contextlib import asynccontextmanager
from pathlib import Path

import polars as pl
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

from phonolex_data.runtime.store import WordStore
from phonolex_generators.csp.reranker.predict import _cached_model

REPO_ROOT = Path(__file__).resolve().parents[3]
DATA_RUNTIME = REPO_ROOT / "data" / "runtime"


@asynccontextmanager
async def lifespan(app: FastAPI):
    print(f"[startup] loading WordStore from {DATA_RUNTIME / 'words.parquet'}…")
    app.state.store = WordStore.from_parquet(DATA_RUNTIME / "words.parquet")
    print(f"[startup] loading selectional.parquet…")
    app.state.sel_df = pl.read_parquet(DATA_RUNTIME / "selectional.parquet")
    print(f"[startup] loading skeletons.parquet…")
    app.state.skeletons_df = pl.read_parquet(DATA_RUNTIME / "skeletons.parquet")
    print(f"[startup] warming reranker model…")
    _cached_model()  # triggers MiniLM + 4-axis booster load
    print(f"[startup] ready.")
    yield


app = FastAPI(lifespan=lifespan)
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # tighten for production
    allow_methods=["*"],
    allow_headers=["*"],
)

from .routes import generate
app.include_router(generate.router, prefix="/api")


@app.get("/health")
async def health():
    return {"status": "ok"}

• [ ] Step 8.3: Rewrite schemas.py

Replace the body with new request/response models:

"""PHON-109: request/response schemas for /api/generate-* endpoints."""
from typing import Literal, Union

from pydantic import BaseModel, Field


# ---------- Constraint schemas (frontend → server) ----------

class ExcludeConstraint(BaseModel):
    type: Literal["exclude"]
    phonemes: list[str]


class IncludeConstraint(BaseModel):
    type: Literal["include"]
    phonemes: list[str]


class BoundConstraint(BaseModel):
    type: Literal["bound"]
    norm: str
    min_value: float | None = None
    max_value: float | None = None


class BoundBoostConstraint(BaseModel):
    type: Literal["bound_boost"]
    norm: str
    min_value: float | None = None
    max_value: float | None = None


class MinpairConstraint(BaseModel):
    type: Literal["contrastive_minpair"]
    phoneme1: str
    phoneme2: str
    position: Literal["initial", "medial", "final", "any"] = "any"
    slots: tuple[str, str] | None = None


class MaxoppConstraint(BaseModel):
    type: Literal["contrastive_maxopp"]
    phoneme1: str
    phoneme2: str
    position: Literal["initial", "medial", "final", "any"] = "any"
    min_sonorant_diff: float = 0.5
    slots: tuple[str, str] | None = None


class MultoppConstraint(BaseModel):
    type: Literal["contrastive_multopp"]
    substitute: str
    targets: list[str]
    n_targets: int = 3
    position: Literal["initial", "medial", "final", "any"] = "any"


Constraint = Union[
    ExcludeConstraint, IncludeConstraint, BoundConstraint, BoundBoostConstraint,
    MinpairConstraint, MaxoppConstraint, MultoppConstraint,
]


# ---------- Request schemas ----------

class GenerateSentencesRequest(BaseModel):
    spec: str
    band: str
    constraints: list[Constraint] = Field(default_factory=list)
    locked_slots: dict[str, str] = Field(default_factory=dict)
    axis_weights: dict[str, float] | None = None
    top_k: int = 8


class GenerateParagraphsRequest(BaseModel):
    spec: str
    band: str
    constraints: list[Constraint] = Field(default_factory=list)
    discourse_subject: str | None = None
    n_sentences: int = 3
    use_pronoun_coref: bool = True
    axis_weights: dict[str, float] | None = None
    top_k: int = 5
    per_sentence_top_k: int = 4
    n_subject_seeds: int = 3


# ---------- Response schemas ----------

class AxisScores(BaseModel):
    naturalness: float
    grammaticality: float
    age_appropriate: float
    coherence: float


class SentenceCandidate(BaseModel):
    sentence: str
    verb: str
    fillers: dict[str, str]
    skeleton: str
    axis_scores: AxisScores
    composite_score: float
    feature_distance: float = 0.0
    sonorant_diff: float = 0.0
    ppmi_total: float = 0.0


class GenerateSentencesResponse(BaseModel):
    candidates: list[SentenceCandidate]
    n_total_candidates: int
    diagnostics: dict = Field(default_factory=dict)


class ParagraphCandidate(BaseModel):
    discourse_subject: str
    sentences: list[SentenceCandidate]
    composite_score: float
    axis_scores: AxisScores
    score: float


class GenerateParagraphsResponse(BaseModel):
    paragraphs: list[ParagraphCandidate]
    n_total_paragraphs: int
    diagnostics: dict = Field(default_factory=dict)

• [ ] Step 8.4: Smoke server boot

cd /Users/jneumann/Repos/PhonoLex && \
  uv run uvicorn packages.generation.server.main:app --host 127.0.0.1 --port 8001 --no-access-log &
sleep 20  # allow cold-start
curl -s http://127.0.0.1:8001/health
kill %1 2>/dev/null

Expected: {"status":"ok"} after ~15s of cold-start logs.

• [ ] Step 8.5: Commit

git add packages/generation/server/main.py packages/generation/server/schemas.py
git commit -m "$(cat <<'EOF'
PHON-109: rewrite server cold-start for CSP + reranker_v2

main.py drops T5Gemma loading; cold-starts WordStore + selectional +
skeletons + reranker model. ~15s cold; warm requests ~1-2s (vs T5Gemma's
~60s cold and per-token decode).

schemas.py defines request/response models for the new three endpoints:
GenerateSentencesRequest/Response, GenerateParagraphsRequest/Response.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
EOF
)"

---

Task 9: /api/generate-sentences route

Files: - Modify: packages/generation/server/routes/generate.py - Modify: packages/generation/server/tests/test_api.py

• [ ] Step 9.1: Inspect existing routes

cat packages/generation/server/routes/generate.py

Note any helpers, error handlers, or shared dependencies.

• [ ] Step 9.2: Rewrite generate.py

Replace body with the three endpoints:

"""PHON-109: /api/generate-sentences, /api/generate-paragraphs, /api/generate-single."""
from fastapi import APIRouter, HTTPException, Request

from phonolex_generators.csp import pair_driven, paragraph
from phonolex_generators.csp.reranker.rerank import rerank_with_axes
from phonolex_generators.csp.constraints import (
    ExcludeConstraint as CExclude, IncludeConstraint as CInclude,
    BoundConstraint as CBound, BoundBoostConstraint as CBoundBoost,
    MinpairConstraint as CMinpair, MaxoppConstraint as CMaxopp,
    MultoppConstraint as CMultopp,
)

from ..schemas import (
    GenerateSentencesRequest, GenerateSentencesResponse, SentenceCandidate, AxisScores,
    GenerateParagraphsRequest, GenerateParagraphsResponse, ParagraphCandidate,
    Constraint as PydConstraint,
)


router = APIRouter()


def _spec_lexicon(store, spec_id: str):
    """Resolve spec_id → frozenset of words. Stub for now;
    move spec_lexicon into phonolex_generators.csp.specs in a follow-up."""
    from phonolex_generators.csp.skeleton import spec_lexicon
    return spec_lexicon(store, spec_id)


def _to_dataclass(c: PydConstraint):
    """Pydantic constraint model → CSP dataclass."""
    if c.type == "exclude":
        return CExclude(phonemes=tuple(c.phonemes))
    if c.type == "include":
        return CInclude(phonemes=tuple(c.phonemes))
    if c.type == "bound":
        return CBound(norm=c.norm, min_value=c.min_value, max_value=c.max_value)
    if c.type == "bound_boost":
        return CBoundBoost(norm=c.norm, min_value=c.min_value, max_value=c.max_value)
    if c.type == "contrastive_minpair":
        return CMinpair(phoneme1=c.phoneme1, phoneme2=c.phoneme2,
                        position=c.position, slots=c.slots)
    if c.type == "contrastive_maxopp":
        return CMaxopp(phoneme1=c.phoneme1, phoneme2=c.phoneme2,
                       position=c.position, min_sonorant_diff=c.min_sonorant_diff,
                       slots=c.slots)
    if c.type == "contrastive_multopp":
        return CMultopp(substitute=c.substitute, targets=tuple(c.targets),
                        n_targets=c.n_targets, position=c.position)
    raise ValueError(f"unknown constraint type: {c.type}")


def _candidate_payload(c: dict) -> SentenceCandidate:
    return SentenceCandidate(
        sentence=c["sentence"],
        verb=c["verb"],
        fillers=c["fillers"],
        skeleton=c["skeleton"],
        axis_scores=AxisScores(**c["axis_scores"]),
        composite_score=c["composite_score"],
        feature_distance=c.get("feature_distance", 0.0),
        sonorant_diff=c.get("sonorant_diff", 0.0),
        ppmi_total=c.get("ppmi_total", 0.0),
    )


@router.post("/generate-sentences", response_model=GenerateSentencesResponse)
async def generate_sentences(req: GenerateSentencesRequest, request: Request):
    state = request.app.state
    spec_words = _spec_lexicon(state.store, req.spec)
    constraints = [_to_dataclass(c) for c in req.constraints]
    try:
        candidates = pair_driven.solve(
            spec_words=spec_words,
            word_df=state.store.df,
            sel_df=state.sel_df,
            pairs_df=state.store.pairs_df,
            skeletons_df=state.skeletons_df,
            band=req.band,
            constraints=constraints,
            locked_slots=req.locked_slots,
            top_k=req.top_k * 4,  # over-fetch for reranker
        )
    except ValueError as e:
        raise HTTPException(status_code=400, detail=str(e))

    if not candidates:
        return GenerateSentencesResponse(candidates=[], n_total_candidates=0)

    n_total = len(candidates)
    ranked = rerank_with_axes(
        candidates,
        is_paragraph=False,
        band=req.band,
        weights=req.axis_weights,
        top_k=req.top_k,
    )
    return GenerateSentencesResponse(
        candidates=[_candidate_payload(c) for c in ranked],
        n_total_candidates=n_total,
    )


@router.post("/generate-single", response_model=SentenceCandidate)
async def generate_single(req: GenerateSentencesRequest, request: Request):
    """Backward-compat alias: top-1 sentence."""
    req_copy = req.model_copy(update={"top_k": 1})
    full = await generate_sentences(req_copy, request)
    if not full.candidates:
        raise HTTPException(status_code=404, detail="no candidates")
    return full.candidates[0]


# /generate-paragraphs added in Task 10

• [ ] Step 9.3: Write API tests

Replace existing test_api.py with:

"""PHON-109: /api/generate-* tests."""
from fastapi.testclient import TestClient
import pytest


@pytest.fixture(scope="module")
def client():
    from packages.generation.server.main import app
    with TestClient(app) as c:
        yield c


def test_health(client):
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}


def test_generate_sentences_minpair(client):
    resp = client.post("/api/generate-sentences", json={
        "spec": "spec1",
        "band": "fineweb_adult",
        "constraints": [
            {"type": "contrastive_minpair", "phoneme1": "d", "phoneme2": "z", "position": "final"}
        ],
        "top_k": 3,
    })
    assert resp.status_code == 200
    data = resp.json()
    assert "candidates" in data
    assert "n_total_candidates" in data
    if data["candidates"]:
        c0 = data["candidates"][0]
        assert "sentence" in c0
        assert "axis_scores" in c0
        assert set(c0["axis_scores"].keys()) == {"naturalness", "grammaticality", "age_appropriate", "coherence"}


def test_generate_single_alias(client):
    resp = client.post("/api/generate-single", json={
        "spec": "spec1",
        "band": "fineweb_adult",
        "constraints": [],
        "top_k": 1,
    })
    # Either 200 with a candidate or 404 if empty — both valid
    assert resp.status_code in (200, 404)


def test_generate_sentences_unknown_constraint_type(client):
    resp = client.post("/api/generate-sentences", json={
        "spec": "spec1",
        "band": "fineweb_adult",
        "constraints": [{"type": "fake_type", "phoneme1": "k"}],
        "top_k": 1,
    })
    assert resp.status_code == 422  # pydantic rejection

• [ ] Step 9.4: Run tests

cd /Users/jneumann/Repos/PhonoLex/packages/generation && \
  uv run python -m pytest server/tests/test_api.py -v

Expected: 4 passed (or 3 + 1 skipped if cold-start is slow).

• [ ] Step 9.5: Commit

git add packages/generation/server/routes/generate.py \
        packages/generation/server/tests/test_api.py
git commit -m "PHON-109: /api/generate-sentences + /api/generate-single alias"

---

Task 10: /api/generate-paragraphs route

Files: - Modify: packages/generation/server/routes/generate.py - Modify: packages/generation/server/tests/test_api.py

• [ ] Step 10.1: Add route handler

Append to generate.py:

def _paragraph_payload(p: dict) -> ParagraphCandidate:
    return ParagraphCandidate(
        discourse_subject=p["discourse_subject"],
        sentences=[_candidate_payload(s) for s in p["sentences"]],
        composite_score=p.get("composite_score", p.get("score", 0.0)),
        axis_scores=AxisScores(**p.get("axis_scores", {ax: 0.0 for ax in ("naturalness", "grammaticality", "age_appropriate", "coherence")})),
        score=p["score"],
    )


@router.post("/generate-paragraphs", response_model=GenerateParagraphsResponse)
async def generate_paragraphs(req: GenerateParagraphsRequest, request: Request):
    state = request.app.state
    spec_words = _spec_lexicon(state.store, req.spec)
    constraints = [_to_dataclass(c) for c in req.constraints]
    spec = paragraph.ParagraphSpec(
        band=req.band,
        constraints=tuple(constraints),
        n_sentences=req.n_sentences,
        discourse_subject=req.discourse_subject,
        use_pronoun_coref=req.use_pronoun_coref,
        n_paragraphs=req.top_k,
        per_sentence_top_k=req.per_sentence_top_k,
        n_subject_seeds=req.n_subject_seeds,
    )
    try:
        paragraphs = paragraph.solve_paragraph(
            spec=spec,
            spec_words=spec_words,
            word_df=state.store.df,
            sel_df=state.sel_df,
            pairs_df=state.store.pairs_df,
            skeletons_df=state.skeletons_df,
        )
    except ValueError as e:
        raise HTTPException(status_code=400, detail=str(e))

    if not paragraphs:
        return GenerateParagraphsResponse(paragraphs=[], n_total_paragraphs=0)

    # Reranker: score paragraphs by joining sentence text + reranker.predict_axes
    n_total = len(paragraphs)
    from phonolex_generators.csp.reranker.predict import predict_axes, composite_score
    for p in paragraphs:
        joined_text = " ".join(s["sentence"] for s in p["sentences"])
        candidate = {"sentence": joined_text, "ppmi_total": p["score"], "feature_distance": 0.0, "sonorant_diff": 0.0}
        ax = predict_axes(candidate, is_paragraph=True, band=req.band)
        p["axis_scores"] = ax
        p["composite_score"] = composite_score(ax, weights=req.axis_weights)
    paragraphs.sort(key=lambda p: -p["composite_score"])
    paragraphs = paragraphs[:req.top_k]

    return GenerateParagraphsResponse(
        paragraphs=[_paragraph_payload(p) for p in paragraphs],
        n_total_paragraphs=n_total,
    )

• [ ] Step 10.2: Add API test

Append to test_api.py:

def test_generate_paragraphs_no_constraint(client):
    resp = client.post("/api/generate-paragraphs", json={
        "spec": "spec1",
        "band": "fineweb_adult",
        "constraints": [],
        "n_sentences": 3,
        "top_k": 2,
    })
    assert resp.status_code == 200
    data = resp.json()
    assert "paragraphs" in data
    if data["paragraphs"]:
        p0 = data["paragraphs"][0]
        assert "discourse_subject" in p0
        assert len(p0["sentences"]) == 3
        assert "composite_score" in p0


def test_generate_paragraphs_multopp(client):
    resp = client.post("/api/generate-paragraphs", json={
        "spec": "spec1",
        "band": "fineweb_adult",
        "constraints": [
            {"type": "contrastive_multopp", "substitute": "t", "targets": ["s", "ʃ"], "n_targets": 2, "position": "initial"}
        ],
        "top_k": 1,
    })
    assert resp.status_code == 200
    data = resp.json()
    if data["paragraphs"]:
        p0 = data["paragraphs"][0]
        assert len(p0["sentences"]) == 3  # 1 sub + 2 targets
        verbs = {s["verb"] for s in p0["sentences"]}
        assert len(verbs) == 1  # shared verb invariant

• [ ] Step 10.3: Run tests

cd /Users/jneumann/Repos/PhonoLex/packages/generation && \
  uv run python -m pytest server/tests/test_api.py -v

Expected: 6 passed (or some skipped on data sparsity).

• [ ] Step 10.4: Commit

git add packages/generation/server/routes/generate.py \
        packages/generation/server/tests/test_api.py
git commit -m "PHON-109: /api/generate-paragraphs route"

---

Task 11: Update Worker proxy

Files: - Modify: packages/web/workers/src/routes/generation.ts - Modify: packages/web/workers/src/__tests__/serverStatus.test.ts - Modify: packages/web/workers/src/lib/serverStatus.ts

The worker proxies /api/generate-* to the backend server. New behavior: backend host comes from a GENERATION_SERVER_URL env var (set later in PHON-109b deployment); when unset, backend is unreachable and we 503 cleanly.

• [ ] Step 11.1: Rewrite generation.ts

Replace body with:

/**
 * Generation proxy — forwards requests to the CSP backend server.
 *
 * PHON-109: replaces the v6/RunPod proxy. The backend is a FastAPI
 * server hosting phonolex_generators.csp + reranker_v2; deployment
 * target is Cloudflare Containers (PHON-109b sibling ticket).
 *
 * Routes:
 *   POST /generate-sentences   → backend /api/generate-sentences
 *   POST /generate-paragraphs  → backend /api/generate-paragraphs
 *   POST /generate-single      → backend /api/generate-single
 *   GET  /server/status        → backend /health
 */

import { Hono } from 'hono';
import type { Env } from '../types';
import { log } from '../lib/logger';

const generation = new Hono<{ Bindings: Env; Variables: { requestId: string } }>();


function backendUrl(env: Env, path: string): string | null {
  if (!env.GENERATION_SERVER_URL) return null;
  return env.GENERATION_SERVER_URL.replace(/\/$/, '') + path;
}


async function proxy(
  c: any,
  method: 'POST' | 'GET',
  path: string,
): Promise<Response> {
  const url = backendUrl(c.env, path);
  if (!url) {
    return c.json({ error: 'Generation backend not configured' }, 503);
  }
  const init: RequestInit = {
    method,
    headers: { 'Content-Type': 'application/json' },
  };
  if (method === 'POST') {
    init.body = await c.req.text();
  }
  try {
    const res = await fetch(url, init);
    const body = await res.text();
    return new Response(body, {
      status: res.status,
      headers: { 'Content-Type': 'application/json' },
    });
  } catch (e) {
    log(c, 'error', 'generation backend fetch failed', { path, error: String(e) });
    return c.json({ error: 'Generation backend unreachable' }, 502);
  }
}


generation.post('/generate-sentences', (c) => proxy(c, 'POST', '/api/generate-sentences'));
generation.post('/generate-paragraphs', (c) => proxy(c, 'POST', '/api/generate-paragraphs'));
generation.post('/generate-single', (c) => proxy(c, 'POST', '/api/generate-single'));
generation.get('/server/status', (c) => proxy(c, 'GET', '/health'));

export default generation;

• [ ] Step 11.2: Update Env type

In packages/web/workers/src/types.ts, replace RUNPOD_API_KEY and RUNPOD_ENDPOINT_ID with:

export interface Env {
  // ... existing fields ...
  GENERATION_SERVER_URL?: string;  // PHON-109: backend FastAPI host (e.g. https://csp.containers.cloudflare.com)
}

Drop the RUNPOD_* fields entirely.

• [ ] Step 11.3: Update serverStatus.ts

The current serverStatus.ts derives a 5-state status from RunPod worker counts. PHON-109's backend is just up-or-down. Simplify:

/**
 * PHON-109: derive 'warm'/'down' from backend /health response.
 */
export type ServerStatus = 'warm' | 'down';


export function deriveServerStatus(healthOk: boolean): ServerStatus {
  return healthOk ? 'warm' : 'down';
}

• [ ] Step 11.4: Update tests

In packages/web/workers/src/__tests__/serverStatus.test.ts, simplify the existing 5-state tests to the 2-state model. The proxy tests in generation.ts aren't currently exercised in unit tests; add a smoke test:

import { describe, it, expect } from 'vitest';
import { deriveServerStatus } from '../lib/serverStatus';


describe('deriveServerStatus', () => {
  it('returns warm when health check ok', () => {
    expect(deriveServerStatus(true)).toBe('warm');
  });
  it('returns down when health check fails', () => {
    expect(deriveServerStatus(false)).toBe('down');
  });
});

• [ ] Step 11.5: Run worker tests

cd /Users/jneumann/Repos/PhonoLex/packages/web/workers && npm test

Expected: all green.

• [ ] Step 11.6: Commit

git add packages/web/workers/src/routes/generation.ts \
        packages/web/workers/src/types.ts \
        packages/web/workers/src/lib/serverStatus.ts \
        packages/web/workers/src/__tests__/serverStatus.test.ts
git commit -m "$(cat <<'EOF'
PHON-109: Worker proxy points at CSP backend (drop RunPod)

generation.ts proxies /generate-{sentences,paragraphs,single} +
/server/status to GENERATION_SERVER_URL (configurable via env). When
unset, returns 503 cleanly.

Drops runpod_url + runpod_headers helpers, RUNPOD_* env vars, and
the 5-state RunPod worker-count parsing. ServerStatus simplifies to
'warm' | 'down' driven by /health.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
EOF
)"

---

Task 12: Retire RunPod artifacts

Files: - Delete: packages/generation/rp_handler.py - Delete: packages/generation/Dockerfile - Delete: packages/generation/server/governor.py - Delete: packages/generation/server/model.py - Delete: packages/generation/server/word_norms.py (consumed by old governor) - Modify: packages/generation/pyproject.toml (drop unused deps) - Modify: wrangler.toml (drop RunPod env vars)

• [ ] Step 12.1: Delete v6 server modules

git rm packages/generation/rp_handler.py
git rm packages/generation/Dockerfile
git rm packages/generation/server/governor.py
git rm packages/generation/server/model.py
git rm packages/generation/server/word_norms.py

If any test imports these, update or delete the test.

• [ ] Step 12.2: Update pyproject.toml

In packages/generation/pyproject.toml, drop deps that were only used by T5Gemma/governor: - torch (if not still used by reranker — check; sentence-transformers needs torch but it's a transitive dep) - transformers (T5Gemma dependency) - Other model-specific deps

Verify lightgbm, sentence-transformers, polars are still listed.

• [ ] Step 12.3: Update wrangler.toml

In packages/web/workers/wrangler.toml, drop the RUNPOD_* env var declarations. Add GENERATION_SERVER_URL as a placeholder env var:

[vars]
GENERATION_SERVER_URL = ""  # set per-environment in PHON-109b

(Or as a secret/binding depending on how the existing wrangler config is structured.)

• [ ] Step 12.4: Run full server suite

cd /Users/jneumann/Repos/PhonoLex/packages/generation && \
  uv run python -m pytest server/tests/ -v

Expected: only the new API tests run + pass; old governor/model tests are gone.

• [ ] Step 12.5: Commit

git add -A packages/generation/ packages/web/workers/wrangler.toml
git commit -m "$(cat <<'EOF'
PHON-109: retire RunPod + T5Gemma artifacts

Deletes:
- packages/generation/rp_handler.py
- packages/generation/Dockerfile (RunPod image; PHON-109b adds Containers Dockerfile)
- packages/generation/server/governor.py (v6 governor)
- packages/generation/server/model.py (T5Gemma loader)
- packages/generation/server/word_norms.py (consumed by old governor)

pyproject.toml drops torch/transformers (T5Gemma deps; sentence-
transformers' torch dep is transitive). wrangler.toml drops RUNPOD_*
env vars; adds GENERATION_SERVER_URL placeholder for PHON-109b.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
EOF
)"

---

Task 13: Final verification

• [ ] Step 13.1: Generators package tests

cd /Users/jneumann/Repos/PhonoLex && \
  uv run python -m pytest packages/generators/tests/ -v

Expected: all CSP tests pass + existing PHON-95 tests still pass.

• [ ] Step 13.2: Server tests

cd /Users/jneumann/Repos/PhonoLex/packages/generation && \
  uv run python -m pytest server/tests/ -v

Expected: API tests pass.

• [ ] Step 13.3: Worker tests

cd /Users/jneumann/Repos/PhonoLex/packages/web/workers && npm test

Expected: all green.

• [ ] Step 13.4: Data layer tests (no regression)

cd /Users/jneumann/Repos/PhonoLex && \
  uv run python -m pytest packages/data/tests/ -q --tb=no

Expected: 209 passed.

• [ ] Step 13.5: End-to-end smoke against running server

cd /Users/jneumann/Repos/PhonoLex && \
  uv run uvicorn packages.generation.server.main:app --host 127.0.0.1 --port 8001 --no-access-log &
sleep 25  # cold-start
curl -s http://127.0.0.1:8001/api/generate-sentences \
  -H 'Content-Type: application/json' \
  -d '{
    "spec": "spec1",
    "band": "fineweb_adult",
    "constraints": [
      {"type": "contrastive_minpair", "phoneme1": "d", "phoneme2": "z", "position": "final"}
    ],
    "top_k": 3
  }' | python -m json.tool
kill %1 2>/dev/null

Expected: 200 response with up to 3 candidates each with axis_scores. Sample: "sentence": "The seed drills in the seas." with composite ~2.65 and 4 axis predictions.

• [ ] Step 13.6: No further commit needed

If all 5 steps pass, PHON-109 is complete.

---

Done

After Task 13 verification, PHON-109 is complete. Branch state: - All CSP code lives in packages/generators/src/phonolex_generators/csp/ - FastAPI server hosts the new endpoints; T5Gemma path retired - Worker proxy targets GENERATION_SERVER_URL (env-configurable) - 3 follow-ups bundled and shipped (locked_slots fix, dedup, plus 2 paragraph_csp workarounds removed) - Spike directory archived (kept for git history; not the canonical path)

Follow-up tickets: - PHON-109b — Cloudflare Containers deployment: build container image, deploy, set GENERATION_SERVER_URL, cutover staging. - PHON-110 — frontend reframe: surface per-axis breakdown + axis weight controls in UI. - PHON-111 — web app continuous-vector parity (filed during PHON-106). - Defer: render-quality polish (PHON-107 follow-up #3).