Saving/Loading document

[arita37]

Hello,

Thanks for the library.
Just wondering if we can load/save from the disk
the computed embedding and models (!)

Thanks !

[raphaelsty]

Hi @arita37, to save the model you can use model.save_pretrained("your-model-name")

To save the embeddings computed you can use Pickle or Joblib :)

[arita37]

Hello, Ok, noted. Usually, I use pandas and parquet to save embeddings Can include meta-data easily (new columns), amd can be consumed in distributed system…. and data pipelines….

…

On Dec 1, 2024, at 1:12, Raphael Sourty ***@***.***> wrote:  Hi @arita37, to save the model you can use model.save_pretrained("your-model-name") To save the embeddings computed you can use Pickle or Joblib :) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.