Skip to content

fix: Sync STT + hand detection on deck change#8

Open
youssofal wants to merge 1 commit into
mainfrom
codex/post-hackathon-upgrades
Open

fix: Sync STT + hand detection on deck change#8
youssofal wants to merge 1 commit into
mainfrom
codex/post-hackathon-upgrades

Conversation

@youssofal
Copy link
Copy Markdown
Owner

Fixes multi-deck switching issues by keeping the hand scanner running on deck share and reloading sherpa-onnx hotwords in-place.

Changes:

  • MainActivity: stop killing HandDetector on deck load; trigger SpeechService hotword reload if running
  • SpeechService: add reloadHotwords() to rebuild OfflineRecognizer with updated deck_hotwords.txt
  • DeckManager: boost deck hotwords via normalized exact match (avoid KNIGHT -> MEGA KNIGHT collisions) and ensure all deck cards are present
  • CardClassifier: more robust deck card -> model class key matching (normalized fallback + logs)

Tested: ./gradlew :app:assembleDebug (BUILD SUCCESSFUL)
Device: N/A (no device connected)

- Keep ML hand scanning running when a new deck is shared

- Reload sherpa-onnx hotwords in SpeechService without a full restart

- Fix deck hotword boosting to avoid substring collisions (KNIGHT vs MEGA KNIGHT)

- Improve deck card key matching to model classes (normalized fallback + warnings)

Tested: ./gradlew :app:assembleDebug (BUILD SUCCESSFUL)

Device: N/A (no device connected)
@youssofal
Copy link
Copy Markdown
Owner Author

Heads up: PR #9 now includes the deck-switch/STT hotword reload changes from this PR (plus Gemini vision improvements). You can merge PR #9 and close this one, or merge this first then PR #9.

Copy link
Copy Markdown
Owner Author

@youssofal youssofal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean fix for the deck switching issues. Key observations:

  1. reloadHotwords() in-place — much better than killing the whole STT service. Keeps recognition responsive during deck changes.

  2. Normalized exact match for hotword boosting — the KNIGHT → MEGA KNIGHT collision fix is important. This was probably causing wrong card plays in mixed decks.

  3. HandDetector stays alive — removing the stop on deck load means detection is continuous, which is the right call for fluid gameplay.

  4. One concern: Does reloadHotwords() block the main thread? If OfflineRecognizer rebuild takes >100ms, might want to move it to a coroutine to avoid UI jank during deck switches.

Overall solid PR. Both this and #9 together make multi-deck gameplay much more reliable.

— Henry (automated review)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant