test_codex

[QingyuanLi1211]

Hello, thank you for such an excellent work, but there is a little question. When you're using test_codex.py, why do you set the prefix and suffix of code['luigi-4'], code['pandas-145'] and code['salt/salt-50958'] individually or is it because the code was written this way just to test these data more easily, as I don't see anything special about these bugs? Looking forward to your reply.

[JohnnyPeng18]

Thanks for your interest! For the specified ones, our function build_prompt have some troubles to get the correct prefixes and suffixes from the original benchmark, so we just directly give them in the code.

[QingyuanLi1211]

Thank you for answering my question. There is another little question that I found 43 bugs in your all_bug_info_bugsinpy.json, but 50 bugs in the benchmarks/bugsinpy, and in your paper, total bugs are 54.

[JohnnyPeng18]

This is because we split different bug locations into different bugs. You can see that there may be more than one entry in the "buglines" field in the JSON file, and that indicates that it contains more than one bugs.