Notice that the param "moe_expert_model_parallelism" hasnt been set in config.json, it is critical,right?
And there are some config params like "ep_size_xx, table_devices_xx,table_experts"，whats the meaning of them?
And looking forward to a example EP bench scripts.
Thanks for your sharing.
Notice that the param "moe_expert_model_parallelism" hasnt been set in config.json, it is critical,right?
And there are some config params like "ep_size_xx, table_devices_xx,table_experts"，whats the meaning of them?
And looking forward to a example EP bench scripts.
Thanks for your sharing.