Unable to reproduce ACDC/Synapse results - seeking guidance

[KyriakiKolpetinou]

Hi,

I've been trying to reproduce the results reported in your paper for the ACDC and Synapse datasets using the nnFormer training pipeline as described, but I'm getting significantly lower Dice scores. My setup:

• nnFormer repo
• Your SegFormer3D baseline integrated into nnFormer
• GPU: nvidia A100
• Patch size used: [14, 160, 160] for ACDC (from nnFormer plans) ((64,128,128) for Synapse)
• I have altered the Segformer network to accept non cubic inputs.
• I've also tried adamW, warmp and scheduling in place of SGD nnFormer uses but still cant get the correct results.

for example for ACDC I'm getting:
Dice_rv 0.8156461043024315
Dice_myo 0.8140759288442456
Dice_lv 0.9206212720308106

which differ a lot from those on the paper. Maybe I'm missing something.

So my question is,
Are there any specific hyperparameters or training configurations beyond the default nnFormer settings that you used? Are the patch sizes im using wrong maybe?

Thank you in advance!

[bnavard]

The nnFormer pipeline includes a wide range of data augmentations and training tricks, which made it fairly complex to port cleanly into our codebase. For the experiments reported in the paper, @srperera ran the ACDC and Synapse benchmarks using the default nnFormer pipeline together with our default SegFormer3D architecture. This required a small, quick modification to nnFormer’s model initialization logic, but otherwise the pipeline was left unchanged. The results we obtained from those runs are what we reported in the paper.

I can’t fully verify whether your changes are bug-free. Unfortunately, I’m also not sure whether @srperera still has access to the exact code used for those experiments, as he and I have since left the lab.

[KyriakiKolpetinou]

Hello,

Thank you for your answer. I, too, used the nnFormer repo unchanged, same pre processing, same lr, batch size, patch size, etc. The splits for train-val I took from the nnFromer json files. This is the model I used, the official Segformer if I'm not mistaken, only change that it accepts non cubic inputs.
segformer3d_baseline.py