[TO_REVIEW] ConditionalTransferableComponentsAdapter Implementation#137
[TO_REVIEW] ConditionalTransferableComponentsAdapter Implementation#137YanisLalou wants to merge 21 commits intoscikit-adaptation:mainfrom
Conversation
| # Define constraint function for Grassmann manifold: W^TW = I_d | ||
| # def grassmann_constraint_function(W): | ||
| # return np.dot(W.T, W) - np.eye(n_components) | ||
|
|
||
| # grassmann_constraint = { | ||
| # "type": "eq", | ||
| # "fun": grassmann_constraint_function, | ||
| # } | ||
|
|
||
| (_, W, _, _), _ = torch_minimize( | ||
| func_torch, (alpha, W, G, H), tol=self.tol, max_iter=1 | ||
| ) | ||
| W = torch.tensor(W, dtype=torch.float64) |
There was a problem hiding this comment.
Here any idea how to enforce the constraint to the torch minimiser ?
There was a problem hiding this comment.
hello, here are a couple of ideas:
-
Use the dual of the optimization problem resulting in adding the constraint to the loss in Pytorch:
min_x f(x) st g(x) = 0 <==> min_{x, lambda} f(x) + lambda * g(x)
here the constraint would be the norm of W^TW - Id -
Another idea could be to generate the matrices directly on the Grassmanian manifold:
a) geomtorch: https://geotorch.readthedocs.io/en/latest/orthogonal/grassmannian.html
b) torch: https://pytorch.org/docs/stable/generated/torch.nn.utils.parametrizations.orthogonal.html
Une autre methode, c'est de parametriser tes fonctions tq ce soit orthogonal avec
- Maybe not directly related, but here is an interesting paper with code to optimize orthogonal matrices:
https://proceedings.mlr.press/v151/ablin22a/ablin22a.pdf (code: https://github.com/pierreablin/landing)
@rflamary @antoinecollas do you have any suggestions?
|
By guaranteeing that W is on the Grassmann manifold we loose performance in the With constraint: Without constraint: In the bench here are the results for constrained/unconstrained CTC: Is it expected or is there an issue ? |
YanisLalou
changed the title
4 participants
Fixes: #96
Paper: https://proceedings.mlr.press/v48/gong16.pdf
Mostly eq 7-8-9-10