smileyface/docs/superpowers/plans/2026-05-30-python-version-support-matrix.md

# Python Version Support Matrix Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Empirically determine and lock in the widest possible range of supported CPython versions (newest → oldest), enforced by a smoke-test suite run across every interpreter via `uv` + `nox`, and guarded in CI by a Gitea Actions matrix.

**Architecture:** `uv` provides every CPython interpreter on demand. A small `pytest` smoke suite (import every module, run every CLI `--help`, load settings) is the pass/fail oracle for "does this version work." `nox` runs that suite against each interpreter in an isolated venv built via `uv`, installing the project through its PEP 517 backend so `pip` resolves the *best dependency versions that work on that interpreter* — which is exactly how we discover the floor. Once discovered, the floor is encoded in `pyproject.toml` (python constraint, classifiers, black targets) and a Gitea Actions workflow mirrors the matrix.

**Tech Stack:** Python 3.9–3.13 (range to be confirmed empirically), Poetry (existing), `uv` (interpreter + venv provisioning), `nox` (matrix runner), `pytest` + `click.testing.CliRunner` (smoke tests), Gitea Actions (CI).

---

## Background facts (verified against the codebase)

These were confirmed during planning — they justify the task ordering. Re-verify if the code has changed.

- **No version-gated syntax** anywhere in `smileyface/`: no `match`/`case`, no runtime `X | Y` unions, no walrus, no 3.11+ stdlib. The code is syntactically 3.8-clean.
- **One version-gated stdlib import:** `smileyface/datalayer/db_ops.py:4` → `from importlib.resources import files` (added in 3.9; needs the `importlib_resources` backport on 3.8).
- **Undeclared runtime dependency (latent bug):** `selenium` is imported at module top level (`import selenium` / `import selenium.webdriver`) in `smileyface/scrape_latest/scrape_ut4pugs.py:1-2` and `smileyface/scrape_latest/scrape_utcc.py:1-2`, but `pyproject.toml` only declares `pydantic-settings`, `click`, `platformdirs`, `sqlparse`. Importing `smileyface.scrape_latest` fails without it. (`requests` is **not** used anywhere — do not add it.)
- **Dependency floors of currently-locked versions:** `pydantic-settings`, `click` (8.2+), `platformdirs` (4.10) all require **3.10+**. Older releases of each support 3.8/3.9, and `pip`/`nox` will select those automatically on older interpreters — which is why per-interpreter resolution (not the shared `poetry.lock`) is used for discovery.
- **No `tests/`, no CI** currently exist. Entry point: `smileyface.py` → `import smileyface; smileyface.start_app()` → `smileyface.cli.cli` (a Click group).
- **CLI `--help` is side-effect-free:** `_make_ctx()` (which builds settings/DB) is only called *inside* command bodies, so invoking any `--help` neither reads a DB nor needs real config. Smoke-testing `--help` is safe.

---

## File Structure

| File | New/Modify | Responsibility |
|------|-----------|----------------|
| `noxfile.py` | Create | Defines the `tests` session parametrized over the Python matrix; uses the `uv` venv backend. |
| `tests/test_imports.py` | Create | Imports every `smileyface.*` submodule; catches syntax/import/missing-dep breakage per interpreter. |
| `tests/test_cli.py` | Create | Invokes `--help` on the root group and every command via `CliRunner`. |
| `tests/test_settings.py` | Create | Verifies `AppSettings` defaults and `SMILEYFACE_*` env parsing. |
| `pyproject.toml` | Modify | Add `selenium` runtime dep; add `pytest`/`nox` dev deps; relax then finalize the python constraint; add classifiers; widen black targets; add pytest config. |
| `smileyface/datalayer/db_ops.py` | Modify (stretch) | `importlib.resources` backport shim, only if pursuing 3.8. |
| `.gitea/workflows/test.yml` | Create | CI matrix mirroring the discovered supported versions. |
| `README.md` | Modify | Document supported versions + how to run the matrix. |
| `CLAUDE.md` | Modify | Add `nox` commands and supported-version note to project guidance. |

---

## Task 1: Bootstrap `uv` and install the candidate interpreters

**Files:** none (environment setup)

- [ ] **Step 1: Confirm `uv` is installed**

Run: `uv --version`
Expected: prints a version (e.g. `uv 0.5.x`). If "command not found", install it:
Run: `curl -LsSf https://astral.sh/uv/install.sh | sh` then restart the shell (or `export PATH="$HOME/.local/bin:$PATH"`).

- [ ] **Step 2: Install every candidate interpreter, newest → oldest**

Run: `uv python install 3.13 3.12 3.11 3.10 3.9 3.8`
Expected: each version downloads/installs (already-present ones are skipped).

- [ ] **Step 3: Verify they are visible to uv**

Run: `uv python list --only-installed`
Expected: lines for cpython-3.13, 3.12, 3.11, 3.10, 3.9, 3.8.

(No commit — this is local environment state.)

---

## Task 2: Add pytest scaffolding and the import smoke test (expected RED)

This test is intentionally expected to **fail first** — it surfaces the undeclared `selenium` dependency before we fix it in Task 3.

**Files:**
- Create: `tests/test_imports.py`
- Modify: `pyproject.toml` (add `pytest` dev dep + pytest config)

- [ ] **Step 1: Add `pytest` to dev dependencies and configure test discovery**

In `pyproject.toml`, under `[tool.poetry.group.dev.dependencies]`, add `pytest`:

```toml
[tool.poetry.group.dev.dependencies]
black = "*"
pre-commit = "*"
isort = "*"
flake8 = "*"
pytest = "*"
```

Then append a pytest config block at the end of the file:

```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-ra"
```

- [ ] **Step 2: Write the import smoke test**

Create `tests/test_imports.py`:

```python
import importlib
import pkgutil

import smileyface


def test_top_level_package_imports():
    importlib.import_module("smileyface")


def test_all_submodules_import():
    errors = []

    def _on_walk_error(name):
        errors.append(f"{name}: failed during package walk")

    module_names = [
        info.name
        for info in pkgutil.walk_packages(
            smileyface.__path__,
            prefix="smileyface.",
            onerror=_on_walk_error,
        )
    ]

    for name in module_names:
        try:
            importlib.import_module(name)
        except Exception as exc:  # noqa: BLE001 - we want every failure, not the first
            errors.append(f"{name}: {exc!r}")

    assert not errors, "Modules failed to import:\n" + "\n".join(errors)
```

- [ ] **Step 3: Run the import test on the newest interpreter and watch it fail**

Run: `uv run --python 3.13 --with . --with pytest pytest tests/test_imports.py -v`
Expected: FAIL — `ModuleNotFoundError: No module named 'selenium'` raised while importing `smileyface.scrape_latest.*`. This confirms the undeclared-dependency bug.

- [ ] **Step 4: Commit the test (red state is intentional and documented in the message)**

```bash
git add tests/test_imports.py pyproject.toml
git commit -m "test: add import smoke test (reveals undeclared selenium dep)"
```

---

## Task 3: Declare the missing runtime dependency (import test goes GREEN)

**Files:**
- Modify: `pyproject.toml` (add `selenium`)

- [ ] **Step 1: Add the undeclared runtime dependency**

In `pyproject.toml`, under `[tool.poetry.dependencies]`, add `selenium`:

```toml
[tool.poetry.dependencies]
python = "^3.13"
pydantic-settings = ">=2.0"
click = ">=8.0"
platformdirs = ">=3.0"
sqlparse = "*"
selenium = ">=4.0"
```

- [ ] **Step 2: Re-run the import test on 3.13 and watch it pass**

Run: `uv run --python 3.13 --with . --with pytest pytest tests/test_imports.py -v`
Expected: PASS — every `smileyface.*` module imports.

- [ ] **Step 3: Commit**

```bash
git add pyproject.toml
git commit -m "fix: declare selenium as a runtime dependency"
```

---

## Task 4: CLI `--help` smoke test

**Files:**
- Create: `tests/test_cli.py`

- [ ] **Step 1: Write the CLI smoke test**

Create `tests/test_cli.py`:

```python
from click.testing import CliRunner

from smileyface.cli import cli


def test_root_help():
    result = CliRunner().invoke(cli, ["--help"])
    assert result.exit_code == 0, result.output
    assert "SmileyFace" in result.output


def test_every_command_help():
    runner = CliRunner()
    for name in cli.commands:
        result = runner.invoke(cli, [name, "--help"])
        assert result.exit_code == 0, f"`{name} --help` failed:\n{result.output}"


def test_scrape_subcommands_help():
    runner = CliRunner()
    scrape_group = cli.commands["scrape"]
    for name in scrape_group.commands:
        result = runner.invoke(cli, ["scrape", name, "--help"])
        assert result.exit_code == 0, f"`scrape {name} --help` failed:\n{result.output}"
```

- [ ] **Step 2: Run it on 3.13**

Run: `uv run --python 3.13 --with . --with pytest pytest tests/test_cli.py -v`
Expected: PASS — root help, all top-level commands, and all `scrape` subcommands return exit code 0.

- [ ] **Step 3: Commit**

```bash
git add tests/test_cli.py
git commit -m "test: add CLI --help smoke tests for every command"
```

---

## Task 5: Settings smoke test

**Files:**
- Create: `tests/test_settings.py`

- [ ] **Step 1: Write the settings test**

Create `tests/test_settings.py`. Passing `_env_file=None` prevents a stray local `.env` from leaking into the assertions:

```python
from smileyface.settings import AppSettings


def test_defaults():
    settings = AppSettings(_env_file=None)
    assert settings.sqlite_filename == "smiles.db"
    assert settings.skip_validate is False
    assert settings.project_dir == ""


def test_env_overrides(monkeypatch):
    monkeypatch.setenv("SMILEYFACE_PROJECT_DIR", "/srv/ut4")
    monkeypatch.setenv("SMILEYFACE_SKIP_VALIDATE", "true")
    settings = AppSettings(_env_file=None)
    assert settings.project_dir == "/srv/ut4"
    assert settings.skip_validate is True
```

- [ ] **Step 2: Run it on 3.13**

Run: `uv run --python 3.13 --with . --with pytest pytest tests/test_settings.py -v`
Expected: PASS — both tests pass (env var override + bool coercion).

- [ ] **Step 3: Run the full suite on 3.13 to confirm everything is green before matrixing**

Run: `uv run --python 3.13 --with . --with pytest pytest -v`
Expected: PASS — all tests across the three files.

- [ ] **Step 4: Commit**

```bash
git add tests/test_settings.py
git commit -m "test: add AppSettings defaults and env-parsing smoke tests"
```

---

## Task 6: Relax the python constraint to enable discovery on older interpreters

`pip` refuses to install a project on an interpreter that violates its `requires-python`. The current `^3.13` blocks installs on 3.8–3.12, so we temporarily widen it. It will be tightened to the discovered floor in Task 10.

**Files:**
- Modify: `pyproject.toml` (python constraint)

- [ ] **Step 1: Widen the python constraint**

In `pyproject.toml`, change the `python` line under `[tool.poetry.dependencies]`:

```toml
python = ">=3.8,<4.0"
```

- [ ] **Step 2: Sanity-check the project still builds/installs on 3.13**

Run: `uv run --python 3.13 --with . --with pytest pytest -v`
Expected: PASS.

- [ ] **Step 3: Commit**

```bash
git add pyproject.toml
git commit -m "build: temporarily widen python constraint to >=3.8 for version discovery"
```

---

## Task 7: Create the `nox` matrix runner

**Files:**
- Create: `noxfile.py`
- Modify: `pyproject.toml` (add `nox` dev dep)

- [ ] **Step 1: Add `nox` to dev dependencies**

In `pyproject.toml`, under `[tool.poetry.group.dev.dependencies]`, add `nox`:

```toml
nox = "*"
```

- [ ] **Step 2: Write `noxfile.py`**

Create `noxfile.py` at the repo root. The `uv` venv backend makes each session build via `uv`, and `session.install(".")` triggers per-interpreter dependency resolution:

```python
import nox

nox.options.default_venv_backend = "uv"
nox.options.reuse_existing_virtualenvs = False

# Newest first. 3.8 is a stretch goal (see Task 9 / Task 8 backport).
PYTHON_VERSIONS = ["3.13", "3.12", "3.11", "3.10", "3.9", "3.8"]


@nox.session(python=PYTHON_VERSIONS)
def tests(session):
    """Install the project + pytest and run the smoke suite on each interpreter."""
    session.install(".")
    session.install("pytest")
    session.run("pytest", "-v")
```

- [ ] **Step 3: Verify nox sees every session**

Run: `uv run --with nox nox --list`
Expected: lists `tests-3.13` through `tests-3.8`.

- [ ] **Step 4: Commit**

```bash
git add noxfile.py pyproject.toml
git commit -m "build: add nox matrix runner for the Python version smoke suite"
```

---

## Task 8: (Stretch) Add the `importlib.resources` backport for 3.8

Do this task **only if** you intend to attempt 3.8 (and only if Task 9 shows dependencies can resolve on 3.8). On 3.9+ the shim is a transparent no-op, so it is safe to land regardless.

**Files:**
- Modify: `smileyface/datalayer/db_ops.py:4`
- Modify: `pyproject.toml` (conditional backport dep)

- [ ] **Step 1: Replace the hard import with a version-tolerant shim**

In `smileyface/datalayer/db_ops.py`, replace line 4:

```python
from importlib.resources import files
```

with:

```python
try:
    from importlib.resources import files  # Python 3.9+
except ImportError:  # Python 3.8
    from importlib_resources import files
```

- [ ] **Step 2: Declare the backport for 3.8 only**

In `pyproject.toml`, under `[tool.poetry.dependencies]`, add an environment-marked dependency:

```toml
importlib-resources = { version = ">=5.0", python = "<3.9" }
```

- [ ] **Step 3: Verify 3.9+ is unaffected**

Run: `uv run --python 3.9 --with . --with pytest pytest tests/test_imports.py -v`
Expected: PASS (uses the stdlib branch; backport not installed).

- [ ] **Step 4: Commit**

```bash
git add smileyface/datalayer/db_ops.py pyproject.toml
git commit -m "compat: fall back to importlib_resources backport on Python 3.8"
```

---

## Task 9: Run the discovery matrix (newest → oldest) and record results

**Files:** none (data collection)

- [ ] **Step 1: Run the full matrix**

Run: `uv run --with nox nox -s tests`
Expected: nox runs `tests-3.13` … `tests-3.8` in turn. Some older sessions may fail at **install** (a dependency such as `pydantic-core` has no wheel / drops support for that interpreter) or at **import/test**. That is the signal we want.

- [ ] **Step 2: Re-run any failed session in isolation to capture the precise cause**

For each version `X` that failed, run: `uv run --with nox nox -s tests-X`
Expected: a clear error — distinguish **resolution/install failure** (dependency floor) from **test failure** (our code). Record the first line of the error.

- [ ] **Step 3: Fill in the results table**

Record outcomes here (replace each `?`):

| Version | Install | Tests | First failure (if any) |
|---------|---------|-------|------------------------|
| 3.13 | ? | ? | |
| 3.12 | ? | ? | |
| 3.11 | ? | ? | |
| 3.10 | ? | ? | |
| 3.9  | ? | ? | |
| 3.8  | ? | ? | |

**The supported floor = the lowest version where both Install and Tests pass.** Note it here: `FLOOR = 3.__`.

- [ ] **Step 4: Commit the recorded results**

```bash
git add docs/superpowers/plans/2026-05-30-python-version-support-matrix.md
git commit -m "docs: record Python version support discovery results"
```

---

## Task 10: Lock in the discovered floor

Use the `FLOOR` value from Task 9. The examples below assume `FLOOR = 3.9`; **substitute the actual floor** in every spot (constraint, classifiers, black targets, and the version list).

**Files:**
- Modify: `pyproject.toml`

- [ ] **Step 1: Set the final python constraint to the floor**

In `pyproject.toml`, set (example for floor 3.9):

```toml
python = ">=3.9,<4.0"
```

- [ ] **Step 2: Trim `noxfile.py` to only the supported versions**

In `noxfile.py`, set `PYTHON_VERSIONS` to exactly the green versions (example for floor 3.9):

```python
PYTHON_VERSIONS = ["3.13", "3.12", "3.11", "3.10", "3.9"]
```

- [ ] **Step 3: Widen black target-version and add classifiers**

In `pyproject.toml`, change the black target to span the supported range (example for floor 3.9):

```toml
[tool.black]
line-length = 120
target-version = ['py39', 'py310', 'py311', 'py312', 'py313']
```

And add a `classifiers` list under `[tool.poetry]` (example for floor 3.9):

```toml
classifiers = [
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: 3.13",
]
```

- [ ] **Step 4: Refresh the lock file for the new range**

Run: `uv run poetry lock`
Expected: succeeds. Note: broadening the range can downgrade shared dev/runtime deps to versions compatible with the floor (e.g. `click` capped below 8.2 because 8.2 requires 3.10). This is the expected cost of wider support.

- [ ] **Step 5: Re-run the trimmed matrix to confirm all-green**

Run: `uv run --with nox nox -s tests`
Expected: every remaining session PASSES.

- [ ] **Step 6: Commit**

```bash
git add pyproject.toml noxfile.py poetry.lock
git commit -m "build: set supported Python floor to 3.9 and widen tooling targets"
```

---

## Task 11: Add the Gitea Actions CI matrix

**Files:**
- Create: `.gitea/workflows/test.yml`

- [ ] **Step 1: Write the workflow**

Create `.gitea/workflows/test.yml`. Set the `python-version` list to exactly the supported versions from Task 10 (example for floor 3.9):

```yaml
name: tests

on:
  push:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4

      - name: Install uv
        uses: astral-sh/setup-uv@v5

      - name: Install Python ${{ matrix.python-version }}
        run: uv python install ${{ matrix.python-version }}

      - name: Create venv
        run: uv venv --python ${{ matrix.python-version }}

      - name: Install project and pytest
        run: uv pip install . pytest

      - name: Run smoke tests
        run: uv run pytest -v
```

- [ ] **Step 2: Validate the YAML locally**

Run: `uv run python -c "import yaml, pathlib; yaml.safe_load(pathlib.Path('.gitea/workflows/test.yml').read_text()); print('yaml ok')"`
Expected: prints `yaml ok`.

- [ ] **Step 3: Commit and push so the runner picks it up**

```bash
git add .gitea/workflows/test.yml
git commit -m "ci: add Gitea Actions Python version matrix"
```

Note: this requires a registered Gitea Actions runner on `git.zavage.net`. If none is registered yet, the workflow is still valid and version-controlled; it will execute once a runner is available. After pushing, confirm the run under the repo's **Actions** tab.

---

## Task 12: Update documentation and final verification

**Files:**
- Modify: `README.md`
- Modify: `CLAUDE.md`

- [ ] **Step 1: Document supported versions and the matrix workflow in README**

In `README.md`, under the `Installation` section, add (example for floor 3.9):

```markdown
## Supported Python Versions

SmileyFace is tested against CPython 3.9 – 3.13. The supported range is
enforced by a smoke-test matrix.

To run the matrix locally (requires [uv](https://docs.astral.sh/uv/)):

    uv python install 3.9 3.10 3.11 3.12 3.13
    uv run --with nox nox -s tests
```

- [ ] **Step 2: Add the matrix commands to CLAUDE.md**

In `CLAUDE.md`, under the `## Commands` section, add before the closing fence of the code block:

```bash
# Run the smoke-test suite on the current interpreter
uv run --with . --with pytest pytest -v

# Run the full Python version matrix
uv run --with nox nox -s tests
```

And change the line `There is no test suite.` to:

```markdown
Smoke tests live in `tests/` (import, CLI `--help`, and settings checks) and run across the supported Python matrix via `nox`.
```

- [ ] **Step 3: Final full verification**

Run: `uv run --with nox nox -s tests`
Expected: every supported session PASSES.

Run: `uv run --python 3.13 --with . --with pytest pytest -v`
Expected: PASS (quick single-interpreter confirmation).

- [ ] **Step 4: Commit**

```bash
git add README.md CLAUDE.md
git commit -m "docs: document supported Python versions and the test matrix"
```

---

## Self-Review

**Spec coverage:**
- "Test each version of Python for support" → Tasks 1, 7, 9 (install interpreters, nox matrix, discovery run).
- "From the newest backwards" → version lists are ordered newest→oldest; discovery records the floor (Task 9).
- "Support as many versions as I can" → Task 6 widens the constraint for discovery; per-interpreter `pip` resolution (Task 7) auto-selects older compatible deps; Task 8 reaches for 3.8 via backport; Task 10 locks the lowest green version.
- "Public-facing distributable app" → classifiers + finalized constraint (Task 10), CI guard (Task 11), user-facing docs (Task 12).

**Placeholder scan:** The only intentional fill-ins are Task 9's results table and the floor value, which are *data to be collected by running the matrix* (not undefined code). Every code/config step contains complete content. Example-floor values (3.9) are explicitly flagged "substitute the actual floor."

**Type/identifier consistency:** `cli` (Click group) and its `.commands` mapping are used consistently in `tests/test_cli.py`; `AppSettings(_env_file=None)` matches the pydantic-settings constructor; `files` import name in the Task 8 shim matches its existing usage in `db_ops.py`; the nox session name `tests` is referenced identically in Tasks 7, 9, 10, 12. `PYTHON_VERSIONS` is the single source of truth trimmed in Task 10.

**Known risk:** If Task 9 shows even 3.9 cannot resolve `pydantic-core` (no compatible wheel), the floor is 3.10 — set every example `3.9` to `3.10` and drop Task 8. This is expected behavior, not a plan defect.