build(deps): update dependency tokenizr to v2

[renovate]

This PR contains the following updates:

Package	Change	Age	Confidence
tokenizr	^1.6.7 → ^2.0.0

---

Release Notes

rse/tokenizr (tokenizr)

v2.2.1

Compare Source

• CLEANUP: cleanup build infrastructure

v2.2.0

Compare Source

• IMPROVEMENT: make alternatives() more robust by using an explicit flag internally
• IMPROVEMENT: improve performance of rule() by using more modern string manipulations
• BUGFIX: ensure that "type" parameter exists when checking in consume()
• BUGFIX: when all alternatives fail, throw the exception of the alternative which went furthest (deepest)
• BUGFIX: be more robust and check alsofor end-of-file situation in peek()
• BUGFIX: fix tab stops in column calculations
• BUGFIX: fix macOS \r character handling in line counting
• BUGFIX: honor stop() situation also in tokenization loop
• BUGFIX: correctly rollback transactions
• BUGFIX: ensure that peek() does not loop infinitely if internal tokenization fails to produce tokens
• BUGFIX: prefer specific rule matches over wildcard rule matches
• CLEANUP: correctly display truncated messages in ParseError.toString()
• CLEANUP: remove RegExp "y" flag runtime-check as it exists since 2015
• CLEANUP: simplify strip() method
• CLEANUP: use Math.min/Math.max for calculations of min/max values
• CLEANUP: fix spellings in README
• UPGRADE: upgrade NPM dependencies

v2.1.2

Compare Source

• UPGRADE: upgrade NPM dependencies
• CLEANUP: cleanup source code

v2.1.1

Compare Source

• UPGRADE: upgrade NPM dependencies
• CLEANUP: cleanup source code

v2.1.0

Compare Source

• IMPROVEMENT: export more types (#​22)
• CLEANUP: make linting more happy by sorting in-place
• UPGRADE: upgrade NPM dependencies

v2.0.3

Compare Source

• UPGRADE: upgrade NPM dependencies
• CLEANUP: switch from any to unknown for the unknown values
• CLEANUP: remove duplicate entries from OxLint config

v2.0.2

Compare Source

• UPGRADE: upgrade NPM dependencies

v2.0.1

Compare Source

• CLEANUP: make OxLint more happy by additional code cleanups

v2.0.0

Compare Source

• IMPROVEMENT: new build infrastructure based on STX, TypeScript, OXLint, Biome, ESLint and Vite
• IMPROVEMENT: converted Tokenizr library source code from JavaScript to TypeScript

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.03%. Comparing base (1d2b69f) to head (5bd0b6d).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #589   +/-   ##
=======================================
  Coverage   86.03%   86.03%           
=======================================
  Files          35       35           
  Lines         859      859           
=======================================
  Hits          739      739           
  Misses        120      120           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.