Poor reproduction performance

[serenity-ruochen]

I highly appreciate your outstanding work. However, when I reproduced this paper, I obtained the following results, which are quite different from the performance of your Table 1 R2R-CE. Could you help me find out what the reason is？ Thank you~

2026-06-14 17:27:36,855 Episodes evaluated: 1839
2026-06-14 17:27:36,855 Average episode steps_taken: 324.125612
2026-06-14 17:27:36,855 Average episode distance_to_goal: 8.485219
2026-06-14 17:27:36,855 Average episode success: 0.132137
2026-06-14 17:27:36,855 Average episode oracle_success: 0.275693
2026-06-14 17:27:36,855 Average episode path_length: 40.218249
2026-06-14 17:27:36,855 Average episode spl: 0.062156
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1839/1839 [21:41:55<00:00, 42.48s/it]
Evaluation completed!

[serenity-ruochen]

I put the inside of the Readme "python third_party/GC-VLN-Server/scripts/quickstart_server/GSAM2 py - port 7000" changed to "python third_party/ModelServer/scripts/GSAM2.py --port 7000", because there is no GC-VLN-Server in your directory, I don't know if this causes the performance is poor.

[bagh2178]

Thank you for your feedback. We found several errors in the installation process (including the third_party/ModelServer/scripts/GSAM2.py file you mentioned), which may affect the environment and consequently impact model performance. We have updated the code repository and verified that the performance can be reproduced. Please update to the latest commit (1beddca) and reinstall the environment following the README. We hope this resolves your issue.