feat: port TextQuery stopword filtering from Python redisvl (#16)

[ymendez-redis]

Closes #16.

Summary

Brings TS TextQuery to Python redisvl parity on stopword behavior:

• stopwords?: string | ReadonlySet<string> | readonly string[] | null on TextQueryConfig, default 'english'.
• Token normalization pre-filter pipeline: trim → strip leading/trailing commas → strip U+201C/U+201D → lowercase (matches query.py:1445).
• Embedded NLTK English list (198 words, BSD-3-Clause Snowball) verbatim from pinned commit 98e1426; raw-bytes SHA-256 recorded for drift detection.
• Public surface: import { stopwords } from 'redis-vl' and import { english } from 'redis-vl/stopwords' (new subpath export).
• THIRD_PARTY_NOTICES.md shipped in the npm tarball (BSD-3 condition Fix HASH vector fetch decoding and score normalization #2).

Phase 1 ships English only; the registry pattern under src/utils/stopwords/ is designed so the 32 remaining NLTK languages drop in as a pure-data follow-up PR without touching TextQuery.

Behavior change (acceptable for 0.1.0-beta.0)

Multi-word queries with default settings now drop English stopwords. Single-word queries are unaffected. The old :::note Minimal port callout in the user-guide already flagged this gap.

'hello, world' now renders as @description:(hello | world) instead of @description:(hello\\, | world) — the comma is normalized away pre-escape, mirroring Python.

To opt out: stopwords: null. To override: pass a string[] or Set<string> (entries stored verbatim — pass lowercase to match the lowercased tokens; Python-parity foot-gun documented in JSDoc and the user-guide).

What's covered

• src/utils/stopwords/{english,registry,resolve,index}.ts — new module
• src/query/text.ts — pipeline rewrite
• src/index.ts — stopwords + StopwordsInput re-export
• package.json — ./stopwords subpath + THIRD_PARTY_NOTICES.md in files
• 4 new unit tests in tests/unit/utils/stopwords/english.test.ts
• 14 new unit tests in tests/unit/utils/stopwords/resolve.test.ts (10 base + 4 prototype-key regression)
• 16 new tests in tests/unit/query/text.test.ts; 1 existing test updated ('hello, world')
• 1 integration test in tests/integration/query-types.test.ts
• website/docs/user-guide/filters-and-queries.md — callout, options table, examples

Test counts: 455 unit / 540 total (was 421 / 506).

Out of scope (separate issues)

• Phase 2: 32 more NLTK languages (mechanical, register + barrel)
• Per-field / per-token text weights from Python
• Server-side STOPWORDS index option

Verification

```
npm run lint # clean (2 pre-existing any warnings in huggingface-vectorizer)
npm run type-check # ok
npm run type-check:tests # ok
npm run test:unit # 455/455
npm run test # 540/540 (incl. integration via testcontainers)
cd website && npm run build # ok
```

Manual smoke:

```ts
import { TextQuery, stopwords } from 'redis-vl';
new TextQuery({ text: 'the quick brown fox', textFieldName: 'd' }).buildQuery();
// → '@d:(quick | brown | fox)'
new TextQuery({ text: 'the quick', textFieldName: 'd', stopwords: null }).buildQuery();
// → '@d:(the | quick)'
new TextQuery({ text: 'foo bar quick', textFieldName: 'd', stopwords: new Set([...stopwords.english, 'foo']) }).buildQuery();
// → '@d:(bar | quick)'
```

Test plan

• CI green on PR
• Review the Snowball / NLTK attribution in `THIRD_PARTY_NOTICES.md`
• Confirm `stopwords` namespace shape is what we want for Phase 2
• Confirm the behavior change on multi-word queries is acceptable pre-1.0

🤖 Generated with Claude Code

[banker]

Thanks @ymendez-redis — this is a thorough port. NLTK attribution + SHA-256 pin + the StopwordsInput shape all look great. A few asks before merge:

1. Drop the planning note from the diff

docs/superpowers/plans/2026-05-20-textquery-stopwords.md shouldn't ship in the repo per AGENTS.md → Files Never Committed ("Any other ad-hoc markdown files during development"). The .gitignore add for .augment/ suggests you're already on board with the rule — please remove this file (and add docs/superpowers/ to .gitignore if you want to keep planning notes locally).

2. Confirm token normalization is Python-parity, not net-new

The new normalizeToken does trim + strip leading/trailing commas + strip typographic quotes + lowercase. Lowercasing is required for the stopword set lookup to work, but the comma/curly-quote stripping is an extra behavior the previous TextQuery didn't have. Could you point at the equivalent in Python redisvl/query/query.py, or call out that this is an intentional improvement over Python? If it's net-new, I'd lean toward dropping it (or scoping it tighter) to keep the parity story clean — comma/quote handling can be a separate small PR.

3. All-stopwords input behavior

Right now new TextQuery({ text: "the and a" }).buildQuery() throws QueryValidationError("text yielded no tokens after normalization and stopword filtering"). What does Python do in this case — throw, emit an empty query string, or something else? Whichever Python does, please match it (and add a test that pins the chosen behavior).

After those three are addressed I'll do a final pass and merge. Heads up: there's an in-flight refactor in #25 that converts TextQuery from implements BaseQuery to extends BaseQuery — I'll coordinate the merge order so you don't have to rebase against it; this PR goes first.