the MLP Projection weights #3
Description
Regarding the MLP Projection weights, I conducted experiments based on your RMLP code. Why do the Projection weights I obtained look the same as the weights of the RLinear linear layer, rather than the chaotic matrix described in your paper?If the Projection weights are similar to the Linear weights, then the product of MLP and Projection will naturally be different from the Linear weights.