Different accuracy when running your spider file

[ahmed98701]

Hello,

First of all, I would like to thank you for your work. It is really beneficial.

Secondly, I downloaded your "dea-sql.txt" file that contains the predicted SPIDER dev sqls.

I tried evaluating that file in a google colab notebook using the official spider test-suite-sql-eval, and these are the results that I got:

Is 76.8 % "EX" on the spider dev dataset expected? Or did I do something wrong? I expected to get 85.4 % based on what is written in the paper:

Here are the steps that I followed in the google colab notebook:

1. !git clone https://github.com/taoyds/test-suite-sql-eval.git

2. !python3 test-suite-sql-eval/evaluation.py
   --gold dev_gold.sql
   --pred dea-sql.txt
   --etype all
   --db /content/drive/MyDrive/text2sql_test_suites/database/database
   --table tables.json

Once again, thank you for your work.