diff --git a/docs/superpowers/plans/2026-05-30-python-version-support-matrix.md b/docs/superpowers/plans/2026-05-30-python-version-support-matrix.md new file mode 100644 index 0000000..3b99559 --- /dev/null +++ b/docs/superpowers/plans/2026-05-30-python-version-support-matrix.md @@ -0,0 +1,633 @@ +# Python Version Support Matrix Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Empirically determine and lock in the widest possible range of supported CPython versions (newest → oldest), enforced by a smoke-test suite run across every interpreter via `uv` + `nox`, and guarded in CI by a Gitea Actions matrix. + +**Architecture:** `uv` provides every CPython interpreter on demand. A small `pytest` smoke suite (import every module, run every CLI `--help`, load settings) is the pass/fail oracle for "does this version work." `nox` runs that suite against each interpreter in an isolated venv built via `uv`, installing the project through its PEP 517 backend so `pip` resolves the *best dependency versions that work on that interpreter* — which is exactly how we discover the floor. Once discovered, the floor is encoded in `pyproject.toml` (python constraint, classifiers, black targets) and a Gitea Actions workflow mirrors the matrix. + +**Tech Stack:** Python 3.9–3.13 (range to be confirmed empirically), Poetry (existing), `uv` (interpreter + venv provisioning), `nox` (matrix runner), `pytest` + `click.testing.CliRunner` (smoke tests), Gitea Actions (CI). + +--- + +## Background facts (verified against the codebase) + +These were confirmed during planning — they justify the task ordering. Re-verify if the code has changed. + +- **No version-gated syntax** anywhere in `smileyface/`: no `match`/`case`, no runtime `X | Y` unions, no walrus, no 3.11+ stdlib. The code is syntactically 3.8-clean. +- **One version-gated stdlib import:** `smileyface/datalayer/db_ops.py:4` → `from importlib.resources import files` (added in 3.9; needs the `importlib_resources` backport on 3.8). +- **Undeclared runtime dependency (latent bug):** `selenium` is imported at module top level (`import selenium` / `import selenium.webdriver`) in `smileyface/scrape_latest/scrape_ut4pugs.py:1-2` and `smileyface/scrape_latest/scrape_utcc.py:1-2`, but `pyproject.toml` only declares `pydantic-settings`, `click`, `platformdirs`, `sqlparse`. Importing `smileyface.scrape_latest` fails without it. (`requests` is **not** used anywhere — do not add it.) +- **Dependency floors of currently-locked versions:** `pydantic-settings`, `click` (8.2+), `platformdirs` (4.10) all require **3.10+**. Older releases of each support 3.8/3.9, and `pip`/`nox` will select those automatically on older interpreters — which is why per-interpreter resolution (not the shared `poetry.lock`) is used for discovery. +- **No `tests/`, no CI** currently exist. Entry point: `smileyface.py` → `import smileyface; smileyface.start_app()` → `smileyface.cli.cli` (a Click group). +- **CLI `--help` is side-effect-free:** `_make_ctx()` (which builds settings/DB) is only called *inside* command bodies, so invoking any `--help` neither reads a DB nor needs real config. Smoke-testing `--help` is safe. + +--- + +## File Structure + +| File | New/Modify | Responsibility | +|------|-----------|----------------| +| `noxfile.py` | Create | Defines the `tests` session parametrized over the Python matrix; uses the `uv` venv backend. | +| `tests/test_imports.py` | Create | Imports every `smileyface.*` submodule; catches syntax/import/missing-dep breakage per interpreter. | +| `tests/test_cli.py` | Create | Invokes `--help` on the root group and every command via `CliRunner`. | +| `tests/test_settings.py` | Create | Verifies `AppSettings` defaults and `SMILEYFACE_*` env parsing. | +| `pyproject.toml` | Modify | Add `selenium` runtime dep; add `pytest`/`nox` dev deps; relax then finalize the python constraint; add classifiers; widen black targets; add pytest config. | +| `smileyface/datalayer/db_ops.py` | Modify (stretch) | `importlib.resources` backport shim, only if pursuing 3.8. | +| `.gitea/workflows/test.yml` | Create | CI matrix mirroring the discovered supported versions. | +| `README.md` | Modify | Document supported versions + how to run the matrix. | +| `CLAUDE.md` | Modify | Add `nox` commands and supported-version note to project guidance. | + +--- + +## Task 1: Bootstrap `uv` and install the candidate interpreters + +**Files:** none (environment setup) + +- [ ] **Step 1: Confirm `uv` is installed** + +Run: `uv --version` +Expected: prints a version (e.g. `uv 0.5.x`). If "command not found", install it: +Run: `curl -LsSf https://astral.sh/uv/install.sh | sh` then restart the shell (or `export PATH="$HOME/.local/bin:$PATH"`). + +- [ ] **Step 2: Install every candidate interpreter, newest → oldest** + +Run: `uv python install 3.13 3.12 3.11 3.10 3.9 3.8` +Expected: each version downloads/installs (already-present ones are skipped). + +- [ ] **Step 3: Verify they are visible to uv** + +Run: `uv python list --only-installed` +Expected: lines for cpython-3.13, 3.12, 3.11, 3.10, 3.9, 3.8. + +(No commit — this is local environment state.) + +--- + +## Task 2: Add pytest scaffolding and the import smoke test (expected RED) + +This test is intentionally expected to **fail first** — it surfaces the undeclared `selenium` dependency before we fix it in Task 3. + +**Files:** +- Create: `tests/test_imports.py` +- Modify: `pyproject.toml` (add `pytest` dev dep + pytest config) + +- [ ] **Step 1: Add `pytest` to dev dependencies and configure test discovery** + +In `pyproject.toml`, under `[tool.poetry.group.dev.dependencies]`, add `pytest`: + +```toml +[tool.poetry.group.dev.dependencies] +black = "*" +pre-commit = "*" +isort = "*" +flake8 = "*" +pytest = "*" +``` + +Then append a pytest config block at the end of the file: + +```toml +[tool.pytest.ini_options] +testpaths = ["tests"] +addopts = "-ra" +``` + +- [ ] **Step 2: Write the import smoke test** + +Create `tests/test_imports.py`: + +```python +import importlib +import pkgutil + +import smileyface + + +def test_top_level_package_imports(): + importlib.import_module("smileyface") + + +def test_all_submodules_import(): + errors = [] + + def _on_walk_error(name): + errors.append(f"{name}: failed during package walk") + + module_names = [ + info.name + for info in pkgutil.walk_packages( + smileyface.__path__, + prefix="smileyface.", + onerror=_on_walk_error, + ) + ] + + for name in module_names: + try: + importlib.import_module(name) + except Exception as exc: # noqa: BLE001 - we want every failure, not the first + errors.append(f"{name}: {exc!r}") + + assert not errors, "Modules failed to import:\n" + "\n".join(errors) +``` + +- [ ] **Step 3: Run the import test on the newest interpreter and watch it fail** + +Run: `uv run --python 3.13 --with . --with pytest pytest tests/test_imports.py -v` +Expected: FAIL — `ModuleNotFoundError: No module named 'selenium'` raised while importing `smileyface.scrape_latest.*`. This confirms the undeclared-dependency bug. + +- [ ] **Step 4: Commit the test (red state is intentional and documented in the message)** + +```bash +git add tests/test_imports.py pyproject.toml +git commit -m "test: add import smoke test (reveals undeclared selenium dep)" +``` + +--- + +## Task 3: Declare the missing runtime dependency (import test goes GREEN) + +**Files:** +- Modify: `pyproject.toml` (add `selenium`) + +- [ ] **Step 1: Add the undeclared runtime dependency** + +In `pyproject.toml`, under `[tool.poetry.dependencies]`, add `selenium`: + +```toml +[tool.poetry.dependencies] +python = "^3.13" +pydantic-settings = ">=2.0" +click = ">=8.0" +platformdirs = ">=3.0" +sqlparse = "*" +selenium = ">=4.0" +``` + +- [ ] **Step 2: Re-run the import test on 3.13 and watch it pass** + +Run: `uv run --python 3.13 --with . --with pytest pytest tests/test_imports.py -v` +Expected: PASS — every `smileyface.*` module imports. + +- [ ] **Step 3: Commit** + +```bash +git add pyproject.toml +git commit -m "fix: declare selenium as a runtime dependency" +``` + +--- + +## Task 4: CLI `--help` smoke test + +**Files:** +- Create: `tests/test_cli.py` + +- [ ] **Step 1: Write the CLI smoke test** + +Create `tests/test_cli.py`: + +```python +from click.testing import CliRunner + +from smileyface.cli import cli + + +def test_root_help(): + result = CliRunner().invoke(cli, ["--help"]) + assert result.exit_code == 0, result.output + assert "SmileyFace" in result.output + + +def test_every_command_help(): + runner = CliRunner() + for name in cli.commands: + result = runner.invoke(cli, [name, "--help"]) + assert result.exit_code == 0, f"`{name} --help` failed:\n{result.output}" + + +def test_scrape_subcommands_help(): + runner = CliRunner() + scrape_group = cli.commands["scrape"] + for name in scrape_group.commands: + result = runner.invoke(cli, ["scrape", name, "--help"]) + assert result.exit_code == 0, f"`scrape {name} --help` failed:\n{result.output}" +``` + +- [ ] **Step 2: Run it on 3.13** + +Run: `uv run --python 3.13 --with . --with pytest pytest tests/test_cli.py -v` +Expected: PASS — root help, all top-level commands, and all `scrape` subcommands return exit code 0. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_cli.py +git commit -m "test: add CLI --help smoke tests for every command" +``` + +--- + +## Task 5: Settings smoke test + +**Files:** +- Create: `tests/test_settings.py` + +- [ ] **Step 1: Write the settings test** + +Create `tests/test_settings.py`. Passing `_env_file=None` prevents a stray local `.env` from leaking into the assertions: + +```python +from smileyface.settings import AppSettings + + +def test_defaults(): + settings = AppSettings(_env_file=None) + assert settings.sqlite_filename == "smiles.db" + assert settings.skip_validate is False + assert settings.project_dir == "" + + +def test_env_overrides(monkeypatch): + monkeypatch.setenv("SMILEYFACE_PROJECT_DIR", "/srv/ut4") + monkeypatch.setenv("SMILEYFACE_SKIP_VALIDATE", "true") + settings = AppSettings(_env_file=None) + assert settings.project_dir == "/srv/ut4" + assert settings.skip_validate is True +``` + +- [ ] **Step 2: Run it on 3.13** + +Run: `uv run --python 3.13 --with . --with pytest pytest tests/test_settings.py -v` +Expected: PASS — both tests pass (env var override + bool coercion). + +- [ ] **Step 3: Run the full suite on 3.13 to confirm everything is green before matrixing** + +Run: `uv run --python 3.13 --with . --with pytest pytest -v` +Expected: PASS — all tests across the three files. + +- [ ] **Step 4: Commit** + +```bash +git add tests/test_settings.py +git commit -m "test: add AppSettings defaults and env-parsing smoke tests" +``` + +--- + +## Task 6: Relax the python constraint to enable discovery on older interpreters + +`pip` refuses to install a project on an interpreter that violates its `requires-python`. The current `^3.13` blocks installs on 3.8–3.12, so we temporarily widen it. It will be tightened to the discovered floor in Task 10. + +**Files:** +- Modify: `pyproject.toml` (python constraint) + +- [ ] **Step 1: Widen the python constraint** + +In `pyproject.toml`, change the `python` line under `[tool.poetry.dependencies]`: + +```toml +python = ">=3.8,<4.0" +``` + +- [ ] **Step 2: Sanity-check the project still builds/installs on 3.13** + +Run: `uv run --python 3.13 --with . --with pytest pytest -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add pyproject.toml +git commit -m "build: temporarily widen python constraint to >=3.8 for version discovery" +``` + +--- + +## Task 7: Create the `nox` matrix runner + +**Files:** +- Create: `noxfile.py` +- Modify: `pyproject.toml` (add `nox` dev dep) + +- [ ] **Step 1: Add `nox` to dev dependencies** + +In `pyproject.toml`, under `[tool.poetry.group.dev.dependencies]`, add `nox`: + +```toml +nox = "*" +``` + +- [ ] **Step 2: Write `noxfile.py`** + +Create `noxfile.py` at the repo root. The `uv` venv backend makes each session build via `uv`, and `session.install(".")` triggers per-interpreter dependency resolution: + +```python +import nox + +nox.options.default_venv_backend = "uv" +nox.options.reuse_existing_virtualenvs = False + +# Newest first. 3.8 is a stretch goal (see Task 9 / Task 8 backport). +PYTHON_VERSIONS = ["3.13", "3.12", "3.11", "3.10", "3.9", "3.8"] + + +@nox.session(python=PYTHON_VERSIONS) +def tests(session): + """Install the project + pytest and run the smoke suite on each interpreter.""" + session.install(".") + session.install("pytest") + session.run("pytest", "-v") +``` + +- [ ] **Step 3: Verify nox sees every session** + +Run: `uv run --with nox nox --list` +Expected: lists `tests-3.13` through `tests-3.8`. + +- [ ] **Step 4: Commit** + +```bash +git add noxfile.py pyproject.toml +git commit -m "build: add nox matrix runner for the Python version smoke suite" +``` + +--- + +## Task 8: (Stretch) Add the `importlib.resources` backport for 3.8 + +Do this task **only if** you intend to attempt 3.8 (and only if Task 9 shows dependencies can resolve on 3.8). On 3.9+ the shim is a transparent no-op, so it is safe to land regardless. + +**Files:** +- Modify: `smileyface/datalayer/db_ops.py:4` +- Modify: `pyproject.toml` (conditional backport dep) + +- [ ] **Step 1: Replace the hard import with a version-tolerant shim** + +In `smileyface/datalayer/db_ops.py`, replace line 4: + +```python +from importlib.resources import files +``` + +with: + +```python +try: + from importlib.resources import files # Python 3.9+ +except ImportError: # Python 3.8 + from importlib_resources import files +``` + +- [ ] **Step 2: Declare the backport for 3.8 only** + +In `pyproject.toml`, under `[tool.poetry.dependencies]`, add an environment-marked dependency: + +```toml +importlib-resources = { version = ">=5.0", python = "<3.9" } +``` + +- [ ] **Step 3: Verify 3.9+ is unaffected** + +Run: `uv run --python 3.9 --with . --with pytest pytest tests/test_imports.py -v` +Expected: PASS (uses the stdlib branch; backport not installed). + +- [ ] **Step 4: Commit** + +```bash +git add smileyface/datalayer/db_ops.py pyproject.toml +git commit -m "compat: fall back to importlib_resources backport on Python 3.8" +``` + +--- + +## Task 9: Run the discovery matrix (newest → oldest) and record results + +**Files:** none (data collection) + +- [ ] **Step 1: Run the full matrix** + +Run: `uv run --with nox nox -s tests` +Expected: nox runs `tests-3.13` … `tests-3.8` in turn. Some older sessions may fail at **install** (a dependency such as `pydantic-core` has no wheel / drops support for that interpreter) or at **import/test**. That is the signal we want. + +- [ ] **Step 2: Re-run any failed session in isolation to capture the precise cause** + +For each version `X` that failed, run: `uv run --with nox nox -s tests-X` +Expected: a clear error — distinguish **resolution/install failure** (dependency floor) from **test failure** (our code). Record the first line of the error. + +- [ ] **Step 3: Fill in the results table** + +Record outcomes here (replace each `?`): + +| Version | Install | Tests | First failure (if any) | +|---------|---------|-------|------------------------| +| 3.13 | ? | ? | | +| 3.12 | ? | ? | | +| 3.11 | ? | ? | | +| 3.10 | ? | ? | | +| 3.9 | ? | ? | | +| 3.8 | ? | ? | | + +**The supported floor = the lowest version where both Install and Tests pass.** Note it here: `FLOOR = 3.__`. + +- [ ] **Step 4: Commit the recorded results** + +```bash +git add docs/superpowers/plans/2026-05-30-python-version-support-matrix.md +git commit -m "docs: record Python version support discovery results" +``` + +--- + +## Task 10: Lock in the discovered floor + +Use the `FLOOR` value from Task 9. The examples below assume `FLOOR = 3.9`; **substitute the actual floor** in every spot (constraint, classifiers, black targets, and the version list). + +**Files:** +- Modify: `pyproject.toml` + +- [ ] **Step 1: Set the final python constraint to the floor** + +In `pyproject.toml`, set (example for floor 3.9): + +```toml +python = ">=3.9,<4.0" +``` + +- [ ] **Step 2: Trim `noxfile.py` to only the supported versions** + +In `noxfile.py`, set `PYTHON_VERSIONS` to exactly the green versions (example for floor 3.9): + +```python +PYTHON_VERSIONS = ["3.13", "3.12", "3.11", "3.10", "3.9"] +``` + +- [ ] **Step 3: Widen black target-version and add classifiers** + +In `pyproject.toml`, change the black target to span the supported range (example for floor 3.9): + +```toml +[tool.black] +line-length = 120 +target-version = ['py39', 'py310', 'py311', 'py312', 'py313'] +``` + +And add a `classifiers` list under `[tool.poetry]` (example for floor 3.9): + +```toml +classifiers = [ + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", +] +``` + +- [ ] **Step 4: Refresh the lock file for the new range** + +Run: `uv run poetry lock` +Expected: succeeds. Note: broadening the range can downgrade shared dev/runtime deps to versions compatible with the floor (e.g. `click` capped below 8.2 because 8.2 requires 3.10). This is the expected cost of wider support. + +- [ ] **Step 5: Re-run the trimmed matrix to confirm all-green** + +Run: `uv run --with nox nox -s tests` +Expected: every remaining session PASSES. + +- [ ] **Step 6: Commit** + +```bash +git add pyproject.toml noxfile.py poetry.lock +git commit -m "build: set supported Python floor to 3.9 and widen tooling targets" +``` + +--- + +## Task 11: Add the Gitea Actions CI matrix + +**Files:** +- Create: `.gitea/workflows/test.yml` + +- [ ] **Step 1: Write the workflow** + +Create `.gitea/workflows/test.yml`. Set the `python-version` list to exactly the supported versions from Task 10 (example for floor 3.9): + +```yaml +name: tests + +on: + push: + pull_request: + +jobs: + test: + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"] + steps: + - uses: actions/checkout@v4 + + - name: Install uv + uses: astral-sh/setup-uv@v5 + + - name: Install Python ${{ matrix.python-version }} + run: uv python install ${{ matrix.python-version }} + + - name: Create venv + run: uv venv --python ${{ matrix.python-version }} + + - name: Install project and pytest + run: uv pip install . pytest + + - name: Run smoke tests + run: uv run pytest -v +``` + +- [ ] **Step 2: Validate the YAML locally** + +Run: `uv run python -c "import yaml, pathlib; yaml.safe_load(pathlib.Path('.gitea/workflows/test.yml').read_text()); print('yaml ok')"` +Expected: prints `yaml ok`. + +- [ ] **Step 3: Commit and push so the runner picks it up** + +```bash +git add .gitea/workflows/test.yml +git commit -m "ci: add Gitea Actions Python version matrix" +``` + +Note: this requires a registered Gitea Actions runner on `git.zavage.net`. If none is registered yet, the workflow is still valid and version-controlled; it will execute once a runner is available. After pushing, confirm the run under the repo's **Actions** tab. + +--- + +## Task 12: Update documentation and final verification + +**Files:** +- Modify: `README.md` +- Modify: `CLAUDE.md` + +- [ ] **Step 1: Document supported versions and the matrix workflow in README** + +In `README.md`, under the `Installation` section, add (example for floor 3.9): + +```markdown +## Supported Python Versions + +SmileyFace is tested against CPython 3.9 – 3.13. The supported range is +enforced by a smoke-test matrix. + +To run the matrix locally (requires [uv](https://docs.astral.sh/uv/)): + + uv python install 3.9 3.10 3.11 3.12 3.13 + uv run --with nox nox -s tests +``` + +- [ ] **Step 2: Add the matrix commands to CLAUDE.md** + +In `CLAUDE.md`, under the `## Commands` section, add before the closing fence of the code block: + +```bash +# Run the smoke-test suite on the current interpreter +uv run --with . --with pytest pytest -v + +# Run the full Python version matrix +uv run --with nox nox -s tests +``` + +And change the line `There is no test suite.` to: + +```markdown +Smoke tests live in `tests/` (import, CLI `--help`, and settings checks) and run across the supported Python matrix via `nox`. +``` + +- [ ] **Step 3: Final full verification** + +Run: `uv run --with nox nox -s tests` +Expected: every supported session PASSES. + +Run: `uv run --python 3.13 --with . --with pytest pytest -v` +Expected: PASS (quick single-interpreter confirmation). + +- [ ] **Step 4: Commit** + +```bash +git add README.md CLAUDE.md +git commit -m "docs: document supported Python versions and the test matrix" +``` + +--- + +## Self-Review + +**Spec coverage:** +- "Test each version of Python for support" → Tasks 1, 7, 9 (install interpreters, nox matrix, discovery run). +- "From the newest backwards" → version lists are ordered newest→oldest; discovery records the floor (Task 9). +- "Support as many versions as I can" → Task 6 widens the constraint for discovery; per-interpreter `pip` resolution (Task 7) auto-selects older compatible deps; Task 8 reaches for 3.8 via backport; Task 10 locks the lowest green version. +- "Public-facing distributable app" → classifiers + finalized constraint (Task 10), CI guard (Task 11), user-facing docs (Task 12). + +**Placeholder scan:** The only intentional fill-ins are Task 9's results table and the floor value, which are *data to be collected by running the matrix* (not undefined code). Every code/config step contains complete content. Example-floor values (3.9) are explicitly flagged "substitute the actual floor." + +**Type/identifier consistency:** `cli` (Click group) and its `.commands` mapping are used consistently in `tests/test_cli.py`; `AppSettings(_env_file=None)` matches the pydantic-settings constructor; `files` import name in the Task 8 shim matches its existing usage in `db_ops.py`; the nox session name `tests` is referenced identically in Tasks 7, 9, 10, 12. `PYTHON_VERSIONS` is the single source of truth trimmed in Task 10. + +**Known risk:** If Task 9 shows even 3.9 cannot resolve `pydantic-core` (no compatible wheel), the floor is 3.10 — set every example `3.9` to `3.10` and drop Task 8. This is expected behavior, not a plan defect.