Skip to content

v0.11.0: public torrent-site catalog with one-click install#1

Merged
masterdraco merged 39 commits intomainfrom
feat/public-torrent-catalog
Apr 20, 2026
Merged

v0.11.0: public torrent-site catalog with one-click install#1
masterdraco merged 39 commits intomainfrom
feat/public-torrent-catalog

Conversation

@masterdraco
Copy link
Copy Markdown
Owner

Summary

  • Curated built-in catalog of 10 public no-account torrent sites (1337x, TorrentGalaxy, LimeTorrents, Nyaa, EZTV, plus 5 substitutes for sites without upstream YAMLs) installable with one click from a new /indexers/catalog page and as an optional onboarding step.
  • Minimal Go-template expander in the Cardigann parser ({{ .Keywords }}, {{ .Config.X }}, {{ if .Keywords }}…{{ else }}…{{ end }}, {{ or }}, inline {{ re_replace }}) so vendored Prowlarr YAMLs actually produce working search URLs.
  • Five new Cardigann filters (urldecode, split, trim, tolower, re_replace) + warn-once log for unknown filters.
  • New scripts/update-catalog.py for syncing/diffing vendored YAMLs against Prowlarr/Indexers.
  • TPB + YTS are NOT in this catalog — both require a native JSON driver (they return JSON, not HTML). Noted as follow-up work.

Test plan

  • cd backend && uv run pytest -q — expect 60 passing
  • cd backend && uv run ruff check src tests && uv run ruff format --check src tests — clean
  • cd web && pnpm check — 0 errors
  • Browse to /indexers/catalog, install a site (e.g. 1337x), verify it appears on /indexers as type=cardigann
  • Go through /onboarding in a fresh install — verify the new "Public torrent sites (optional)" step appears between indexer and AI steps
  • Run a real search against an installed catalog site to verify Go template expansion produces valid URLs

Design + plan

  • Spec: docs/superpowers/specs/2026-04-20-public-torrent-site-catalog-design.md
  • Plan: docs/superpowers/plans/2026-04-20-public-torrent-site-catalog.md

Follow-up work (not blocking)

  • Capture HTML fixtures for the remaining 9 slugs (harness already shipped; Nyaa fixture included as worked example). Tracked in TODO.md.
  • Native JSON driver for TPB and YTS (out of scope for this PR — HTML-scraping parser can't drive them).
  • regexp filter missing re.error catch (pre-existing; not introduced by this branch).

🤖 Generated with Claude Code

masterdraco and others added 30 commits April 20, 2026 11:05
Adds a design document for a curated, built-in catalog of 12 public
no-account torrent sites (TPB, 1337x, Nyaa, TorrentGalaxy, …). Ships
the parser machinery already on disk — the gap is UX. The spec covers
catalog layout, vendoring from Prowlarr-indexers, parser extensions
driven by real per-site fixture tests, a dedicated /indexers/catalog
page, and an optional onboarding step.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bite-sized task decomposition with code + test + commit steps for each
chunk: Alembic migration, catalog registry + vendoring script, five
new Cardigann parser filters (TDD), two new API endpoints, catalog
page, /indexers shortcut, onboarding step, and a parametrized fixture
test harness. References the spec committed in d676ec4.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The torrentgalaxyclone.yml definition covers the .one/.info/.space
domains, not .to/tgx.rs. Registry mirror list was stale.
Also fix import sort order in main.py and type: ignore comment in catalog.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds backend/tests/test_catalog_fixtures.py, parametrized over any
backend/tests/fixtures/catalog/<slug>-search.html files present. Zero
parametrizations for now — no fixture was captured. TPB was attempted via
tpb.party but the vendored YAML uses the apibay.org JSON API, not HTML
parsing, so captured HTML cannot be exercised by the row extractor.
Similarly, thepiratebay.org itself returns a JS-rendered shell page
(<5 KB). The harness is still wired correctly and will activate as soon
as an HTML-based site fixture is dropped in the fixtures directory.

Also adds TODO.md with a reminder to capture fixtures for the remaining
11 catalog slugs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a "Catalog (public sites, one click)" section to 03-indexers.md
immediately before "Priority and ordering", covering how to install from
the catalog, what the onboarding wizard does, and how vendored YAMLs are
updated.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
masterdraco and others added 9 commits April 20, 2026 16:34
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…, params, selectors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…e tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…lates

Fixes torrentgalaxy and bitsearch search URLs which used template
constructs the initial expander did not cover. Sanity sweep confirms
all 10 catalog sites now produce URLs without unresolved template
tokens.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@masterdraco masterdraco merged commit 72e3a8b into main Apr 20, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant