I was curious to test some flags against each other with Wasmtime to determine their effect on performance today, so I passed the same *.so as the -e flag to two collection runs with different --engine-flags values. When running effect-size, however, it failed with:
Error: Can only test significance between exactly two different engines. Found 1 different engines.
I then renamed the same *.so to two different files and the comparison came out ok, so this might be a case where factoring in the flags to the uniqueness calculating and/or printing later on would help.
I was curious to test some flags against each other with Wasmtime to determine their effect on performance today, so I passed the same
*.soas the-eflag to two collection runs with different--engine-flagsvalues. When runningeffect-size, however, it failed with:I then renamed the same
*.soto two different files and the comparison came out ok, so this might be a case where factoring in the flags to the uniqueness calculating and/or printing later on would help.