Skip to content

Conversation

@0xrushi
Copy link
Contributor

@0xrushi 0xrushi commented Jan 22, 2026

Add Training and Evaluation Results Documentation

This PR adds comprehensive results documentation to the Whisper sneeze adapter fine-tuning README, documenting the model's performance after training.

Changes

  • Added Results Section: Comprehensive documentation of training and evaluation metrics
    • Training loss progression table showing convergence over 200 steps
    • Final training metrics (loss: 0.001, validation loss: 0.002856, WER: 0.0%)
    • Evaluation results on test set (WER: 32.17%, Sneeze Recall: 50%, False Positives: 0)
    • Analysis of missed sneezes with specific examples
    • Performance analysis and insights

Key Results

  • Training: Achieved 0.0% WER on validation set after 200 training steps (probably overfit)
  • Evaluation: 50% sneeze recall (2/4 detected) with zero false positives
  • Efficiency: fine-tuned with only 13 training examples

Impact

This documentation provides transparency into the model's performance and demonstrates that effective fine-tuning is possible with minimal data using LoRA adapters, making it easier for others to understand the project's outcomes and reproducibility.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 22, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • 🔍 Trigger a full review
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@0xrushi 0xrushi changed the title docs: Add sneeze detection model to finetune whisper docs: Add sneeze transcription model to finetune whisper Jan 22, 2026
@0xrushi 0xrushi changed the title docs: Add sneeze transcription model to finetune whisper temp do not merge: Add sneeze transcription model to finetune whisper Jan 22, 2026
@0xrushi 0xrushi marked this pull request as draft January 22, 2026 05:37
- Updated the `maybe_anomaly` field in the `Conversation` model to include 'rejected' as a possible status.
- Implemented logic to mark transcripts as handled in the database when rejected, preventing them from reappearing in the events queue.
- Added a new environment variable to enable or disable the user loop modal in the frontend.
- Refactored the `UserLoopModal` component to fetch events from the backend and conditionally display based on the new environment variable.
- Improved error handling in event fetching to ensure robustness.
- Updated tests to verify the correct handling of rejected transcripts and their impact on the events queue.
- Updated user loop integration tests to improve structure and clarity.
- Added new keywords for marking test conversations as deleted in the helper functions.
- Enhanced test cases to ensure proper handling of rejected and accepted swipe flows.
- Improved documentation for test cases and keywords to reflect recent changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant