-
Notifications
You must be signed in to change notification settings - Fork 90
Open
Description
Hi Loc-Bench creators,
We have our own implementation of extracting the modified nodes from a given gold patch and we were converting our format to map to Loc-Bench edit_functions format.
We have hit an issue for these 3 instances, where the edit_functions values are incorrect for the Loc-Bench czlll/Loc-Bench_V1 dataset on Huggingface. For these 3 instances, these files and functions/modules are missing from edit_functions
Mismatch instance_id: AzureAD__microsoft-authentication-library-for-python-454
missing_from_ours (0): []
extra_in_ours (3): ['msal/application.py', 'msal/oauth2cli/assertion.py', 'msal/oauth2cli/oauth2.py']
================================================================================
Mismatch instance_id: NVIDIA__NeMo-9100
missing_from_ours (0): []
extra_in_ours (1): ['nemo/collections/asr/parts/submodules/ctc_greedy_decoding.py']
================================================================================
Mismatch instance_id: matrix-org__synapse-8744
missing_from_ours (0): []
extra_in_ours (1): ['synapse/handlers/receipts.py']
I noticed another open issue on GH here: #20 which points out the same issue.
Could you please take a look? Please let me know if you need additional information.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels