Skip to content

Incorrect edit_functions values for some instances #24

@jatinganhotra

Description

@jatinganhotra

Hi Loc-Bench creators,

We have our own implementation of extracting the modified nodes from a given gold patch and we were converting our format to map to Loc-Bench edit_functions format.

We have hit an issue for these 3 instances, where the edit_functions values are incorrect for the Loc-Bench czlll/Loc-Bench_V1 dataset on Huggingface. For these 3 instances, these files and functions/modules are missing from edit_functions

Mismatch instance_id: AzureAD__microsoft-authentication-library-for-python-454
  missing_from_ours (0): []
  extra_in_ours   (3): ['msal/application.py', 'msal/oauth2cli/assertion.py', 'msal/oauth2cli/oauth2.py']
================================================================================
Mismatch instance_id: NVIDIA__NeMo-9100
  missing_from_ours (0): []
  extra_in_ours   (1): ['nemo/collections/asr/parts/submodules/ctc_greedy_decoding.py']
================================================================================
Mismatch instance_id: matrix-org__synapse-8744
  missing_from_ours (0): []
  extra_in_ours   (1): ['synapse/handlers/receipts.py']

I noticed another open issue on GH here: #20 which points out the same issue.

Could you please take a look? Please let me know if you need additional information.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions