About aux loss

[AndPuQing]

Hello, I noticed that the aux loss is used in the paper, where is the rmi loss used?

thanks for your work

[mehtadushy]

If I remember correctly, we attach both losses to a single head. There isn't a separate auxiliary head used for aux loss, which is a departure from how it is typically done.
You can experiment though with multiple heads. Our purpose in removing the aux head was to simplify the architecture used in training to eventually make NAS less fussy