Need "bert_table_tokenized.pkl" to evaluate retriever

[km9mn]

Hi

I want to evaluate trained retriever.
I got this error message running retriever_eval.py.

Traceback (most recent call last):
  File "./src/retriever_eval.py", line 210, in <module>
    main()
  File "./src/retriever_eval.py", line 200, in main
    index = build_faiss_index(config, args, accelerator)
  File "./src/retriever_eval.py", line 57, in build_faiss_index
    table_dataloader = get_table_dataloader(config)
  File "/home/kyumin/project/table/Open_WikiTable/src/dataloader.py", line 320, in get_table_dataloader
    data = tokenized_table(config)
  File "/home/kyumin/project/table/Open_WikiTable/src/dataloader.py", line 305, in __init__
    self.tokenized = torch.load(path)
  File "/home/kyumin/anaconda3/envs/openwiki/lib/python3.8/site-packages/torch/serialization.py", line 699, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/home/kyumin/anaconda3/envs/openwiki/lib/python3.8/site-packages/torch/serialization.py", line 230, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/home/kyumin/anaconda3/envs/openwiki/lib/python3.8/site-packages/torch/serialization.py", line 211, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: './data/augmented/bert_table_tokenized.pkl'

Where can I get it?

Thank you in advance.

[yuxiang-guo]

@km9mn Hi, have you solved this problem? I got a similar error when running retriever_train.py
File "./src/retriever_train.py", line 148, in
main()
File "./src/retriever_train.py", line 144, in main
trainer(config,args)
File "./src/retriever_train.py", line 61, in trainer
train_dataloader = get_retriever_dataloader(question_tokenizer, config)
File "/home/yxguo/Open_WikiTable-main/Open_WikiTable-main/src/dataloader.py", line 182, in get_retriever_dataloader
train_dataset.to_json(path)
File "/home/yxguo/anaconda3/envs/myditto/lib/python3.7/site-packages/pandas/core/generic.py", line 2571, in to_json
storage_options=storage_options,
File "/home/yxguo/anaconda3/envs/myditto/lib/python3.7/site-packages/pandas/io/json/_json.py", line 131, in to_json
path_or_buf, "w", compression=compression, storage_options=storage_options
File "/home/yxguo/anaconda3/envs/myditto/lib/python3.7/site-packages/pandas/io/common.py", line 707, in get_handle
newline="",
FileNotFoundError: [Errno 2] No such file or directory: './data/augmented/reader_train.json'