Add config files to experiment on the performance between dense and MOE blocks in attention layers#18

Adefioye wants to merge 4 commits into

chrisjmccormick:mainchrisjmccormick/shared-subspaces:mainfrom

Adefioye:first-exp-moeAdefioye/shared-subspaces:first-exp-moeCopy head branch name to clipboard