feat(PF-4339,PF-4346,PF-4343): add scanner skill evals

[jpuzz0]

Summary

• Add eval configs and test cases for 3 scanner skills (Batch 1 of PF-4230 eval coverage plan)
• pf-raw-colors-scan: 3 cases — raw hex/rgb/hsl detection, token declaration exception, already-tokenized negative control
• pf-import-checker: 3 cases — invalid charts import, invalid chatbot import, valid PF6 imports negative control
• pf-class-migration-scanner: 3 cases — legacy versioned classes, legacy token patterns, valid PF6 code negative control
• All judges use deterministic check type (Python regex/string matching)
• Negative-control judges check for absence of violation markers (Kent's testing feedback applied)

Closes: PF-4339
Closes: PF-4346
Closes: PF-4343

Summary by CodeRabbit

• New Features

  • Added a new migration skill example for breaking-change detection.
  • Expanded evaluation coverage with additional PatternFly class, import, and color-check scenarios.

• Documentation

  • Updated skill and plugin guides to better describe content review and migration capabilities.
  • Refreshed the skills count badge to reflect the current total.

• Bug Fixes

  • Clarified guidance so the right plugin examples are easier to find and use.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 11c84778-1c87-4293-97e6-08279d2d434f

📥 Commits

Reviewing files that changed from the base of the PR and between 56cda6c and 1c32b49.

📒 Files selected for processing (35)

• CONTRIBUTING-SKILLS.md
• PLUGINS.md
• README.md
• eval/pf-class-migration-scanner/cases/legacy-token-patterns/_legacy-overrides.scss
• eval/pf-class-migration-scanner/cases/legacy-token-patterns/annotations.yaml
• eval/pf-class-migration-scanner/cases/legacy-token-patterns/input.yaml
• eval/pf-class-migration-scanner/cases/legacy-versioned-classes/LegacyNavbar.tsx
• eval/pf-class-migration-scanner/cases/legacy-versioned-classes/annotations.yaml
• eval/pf-class-migration-scanner/cases/legacy-versioned-classes/input.yaml
• eval/pf-class-migration-scanner/cases/valid-pf6-code/ModernCard.tsx
• eval/pf-class-migration-scanner/cases/valid-pf6-code/annotations.yaml
• eval/pf-class-migration-scanner/cases/valid-pf6-code/input.yaml
• eval/pf-class-migration-scanner/eval.yaml
• eval/pf-import-checker/cases/invalid-charts-import/ChartsDashboard.tsx
• eval/pf-import-checker/cases/invalid-charts-import/annotations.yaml
• eval/pf-import-checker/cases/invalid-charts-import/input.yaml
• eval/pf-import-checker/cases/invalid-chatbot-import/AssistantPanel.tsx
• eval/pf-import-checker/cases/invalid-chatbot-import/annotations.yaml
• eval/pf-import-checker/cases/invalid-chatbot-import/input.yaml
• eval/pf-import-checker/cases/valid-pf6-imports/SettingsPage.tsx
• eval/pf-import-checker/cases/valid-pf6-imports/annotations.yaml
• eval/pf-import-checker/cases/valid-pf6-imports/input.yaml
• eval/pf-import-checker/eval.yaml
• eval/pf-raw-colors-scan/cases/already-tokenized/_dashboard.scss
• eval/pf-raw-colors-scan/cases/already-tokenized/annotations.yaml
• eval/pf-raw-colors-scan/cases/already-tokenized/input.yaml
• eval/pf-raw-colors-scan/cases/raw-hex-in-styles/_card-styles.scss
• eval/pf-raw-colors-scan/cases/raw-hex-in-styles/annotations.yaml
• eval/pf-raw-colors-scan/cases/raw-hex-in-styles/input.yaml
• eval/pf-raw-colors-scan/cases/token-declaration-exception/_token-definitions.scss
• eval/pf-raw-colors-scan/cases/token-declaration-exception/annotations.yaml
• eval/pf-raw-colors-scan/cases/token-declaration-exception/input.yaml
• eval/pf-raw-colors-scan/eval.yaml
• plugins/migration/README.md
• plugins/pf-workshop/README.md

---

📝 Walkthrough

Walkthrough

Adds three new evaluation suites (pf-class-migration-scanner, pf-import-checker, pf-raw-colors-scan) under eval/, each with test fixtures (TSX/SCSS), annotations.yaml, input.yaml, and eval.yaml judge configurations. Also updates documentation: README.md skills badge count, PLUGINS.md and plugin README descriptions, and CONTRIBUTING-SKILLS.md plugin table.

Changes

Evaluation Suites

Layer / File(s)	Summary
pf-class-migration-scanner eval suite eval/pf-class-migration-scanner/cases/*, eval/pf-class-migration-scanner/eval.yaml	Adds legacy token/class fixtures (SCSS, TSX), a valid PF6 code fixture, per-case annotations/input configs, and an eval.yaml defining five judges checking legacy pattern detection, PF6 replacement suggestions, confidence levels, and false-positive avoidance.
pf-import-checker eval suite eval/pf-import-checker/cases/*, eval/pf-import-checker/eval.yaml	Adds invalid-import fixtures (charts, chatbot) and a valid-imports fixture, per-case annotations/input configs, and an eval.yaml defining judges that check flagged invalid imports, suggested correct imports, and absence of false positives.
pf-raw-colors-scan eval suite eval/pf-raw-colors-scan/cases/*, eval/pf-raw-colors-scan/eval.yaml	Adds SCSS fixtures for raw-hex colors, already-tokenized styles, and token-declaration exceptions, per-case annotations/input configs, and an eval.yaml defining judges checking raw color detection, PF token suggestions, and exception handling.

Documentation Updates

Layer / File(s)	Summary
Skill listing and description updates README.md, CONTRIBUTING-SKILLS.md, PLUGINS.md, plugins/migration/README.md, plugins/pf-workshop/README.md	Updates the skills badge count (31 to 32), adds pf-react-breaking-changes to the migration plugin's example skills and bullet list, and shortens the pf-content-review description.

Estimated code review effort: 2 (Simple) | ~15 minutes

Possibly related issues

• Eval: pf-raw-colors-scan #120: Adds the pf-raw-colors-scan eval suite and fixtures matching this issue's request to build that evaluation.
• Eval: pf-class-migration-scanner #124: Adds the pf-class-migration-scanner eval suite and related migration-plugin docs matching this issue's evaluation request.
• Eval: pf-import-checker #127: Adds the pf-import-checker eval suite and fixtures for PF6 import-path validation matching this issue's eval request.

Possibly related PRs

• patternfly/ai-helpers#6: Adds the pf-class-migration-scanner evaluation cases/config corresponding to this PR's introduction of the same scanner skill.
• patternfly/ai-helpers#109: This PR updates pf-content-review documentation text, tying to the PR that added the content plugin's pf-content-review agent/skill.
• patternfly/ai-helpers#116: This PR's docs update adds migration:pf-react-breaking-changes to the skills listing, matching the PR that introduced that skill.

Suggested reviewers: dlabaj

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}