In the doc of dense_scores, it is mentioned that: torch.mean is dedicated to training
|
func |
|
Either torch.sum or torch.mean. torch.mean is dedicated to training and |
|
torch.sum is dedicated to inference. |
However, in your training function of SparseEmb, you use torch.sum instead. So which one is preferable for training? I guess it should be torch.mean?
In the doc of
dense_scores, it is mentioned that:torch.mean is dedicated to trainingneural-cherche/neural_cherche/utils/dense_scores.py
Lines 138 to 140 in 2df0214
However, in your training function of SparseEmb, you use
torch.suminstead. So which one is preferable for training? I guess it should betorch.mean?neural-cherche/neural_cherche/train/train_sparse_embed.py
Line 129 in 2df0214