The implementation of CLIP-NCE inconsistent with paper?

[elliottzheng]

According to the paper, the CLIP-NCE loss is like

there should be two positive terms in the denominator, while in the code

CF-CLIP/criteria/nce_loss.py

Line 99 in 58c0413

return self.InfoNCE(query.repeat(2,1), torch.cat([pos1_direction,pos2_direction], 0), neg_direction)

if I understand correctly, there will be only one positive term in the denominator (as the two positive is separately considered as different batch), which seems inconsistent.

[yingchen001]

Hi Elliott,

Thank you very much for pointing out this issue, yes we treat them separately in batch dim and we will correct the equation as below on the camera-ready version of our paper. Meanwhile, we will conduct experiments on our model with the implementation of above equation.