Pre-training

[MananSuri27]

Can you provide documentation to perform the pre-training on custom datasets?
How is the data formatted and what are the appropriate parameters for the scripts in the ./pre_training directory?

Thank you!

[beyondguo]

Thanks for your interest!

We provide example code at:

• pre_training/prepare_genius_pretrain_data.py, for data preparation
• pre_training/genius_pretrain.py, for training on the processed data

There aren't too many parameters for tuning, you can use the parameters in the example. (may change the batch-size or num_workers to fit in your hardware.)

We will write documentation for custom pre-training, maybe next week. Before that, please try the example code with a few modifications. If you have problem, feel free to contact me again :)