Dialo_Training

First steps, practice, learning fixes

Quick Debug: Skipped W&B Logging

Hugging Face defaults to report_to="wandb" for metrics. I set report_to=None to keep it local — no external deps, faster prototyping. Result: Still calling for a W&B API key.

Debug Win: W&B Wouldn't Die

Colab auto-started W&B despite report_to=None.
I used os.environ["WANDB_DISABLED"] = "true" + runtime restart.
Result: Local training. No external deps.

Fix I Applied: pad_token = eos_token

GPT models don't have a padding token by default.
I set tokenizer.pad_token = tokenizer.eos_token to enable batch training.
Result: Garbage in = Gibberish out.

Final Fix: Added labels for Causal LM Training

DialoGPT is autoregressive — needs labels = input_ids to compute loss.
Without it: logits only.
I fixed tokenization + bumped epochs to 5 for 10-example dataset.
Result: Bot memorized my examples, but forgot how to respond to anything except exact training.

Overfitting? Been There.

5 epochs on 10 examples → bot went silent.
Fix: Dropped to 1 epoch + used chat_with_tuned_bot rather than chat_with_bot
Result: chat_with_tuned_bot not defined.
Lesson: — avoid overfitting chat_with_tuned_bot with temperature=0.9. Result: Training on a small dataset of full conversation examples rather than next-token prediction doesn't work for this task.

End of Project