LayerSkip Code Checklist

[mostafaelhoushi]

Here is a list of tasks and ideas... they are not all necessary. Also not sure after which task to open source... we can opensource now.

• Use HuggingFace datasets for CNN/DM, XSUM, HumanEval
• Make num_shots, seeds become arguments
• Add License
• Add unit tests #7
• Sweep on optimal early_exit_layer and num_speculations
• Get correctness.py to Work
• Implement sampling with temperature
• Address minor TODO comments in code
• Refactor arguments across scripts
• Refactor generate.py
• Clean up and review README #6
• Implement Text Streaming for Output
• Implement Text Correction Streaming for Self-Speculation
• Implement training using Early Exit Loss
• Implement training using Layer Dropout

Cc @AkshatSh