added cosine matching loss#2482
Conversation
clessig
left a comment
There was a problem hiding this comment.
Thanks for the implementation. Can we generalise the PR slightly to have a LossLatent (I think it's a regularisation term since it does not involve targets but this is a minor point), with the cosine being a specific implementation (RMSE or other losses are possible as well). We will also need to best structure the part in the model but let's first to this restructuring.
@MatKbauer : (I don't think so since this uses that target_aux_calculator but double-checking)
@sophie-xhonneux : for me this is more a regularisation term since no target are involved. Any opinion on this?
structured the model forward function renmaed LossLatentCosineMatching to LossLatent removed target_aux_caluclator from the config
a7dbbf0 to
50e5e32
Compare
Thanks for the comments! I have implemented few changes you proposed. Now I have removed targets and changed the naming to LatentLoss. |
Description
Added cosine matching loss which is used for constraining cosine similarity between consecutive forecasting latents
Issue Number
Link the Issue number this change addresses:
Closes #2481