Reproduction of the result.

[IzhiSu]

Dear authors,

I am not able to reproduce the results reported in the paper using the given config at config/mbptrack_kitti_ped_cfg.yaml.

I wonder if the same config used for generating the results in the paper is provided. Or is there anything else we can take note of the reproduce the results?

Thanks!

[slothfulxtx]

Hi,
I've uploaded all my code and checkpoints on github, thus it should be easy to reproduce our results. I've just tested my code and checkpoint on our server

python main.py configs/mbptrack_kitti_ped_cfg.yaml --phase test --resume_from pretrained/mbptrack_kitti_ped.ckpt

The results are listed below

Notably, this server is also occupied by my colleagues for other tasks, thus the runtime metric is not accurate.

[IzhiSu]

Thanks for your reply! When I reproduced the category such as ’Pedestrian‘, the experimental results were 66.25/91.10. Could you please provide me with specific configurations to help me reproduce this result? Looking forward to your reply!

[slothfulxtx]

Actually, MBPTrack reuses previous prediction results when the tracked target is missing. If the prediction of MBPTrack is not accurate enough, it may lose targets for the following frames. (You can find some tracklets of extremely low quality in KITTI.) Thus the performance of MBPTrack is not very robust and has a minor fluctuation. I suggest you can train our model from scratch to reproduce the performance.

[ujaejoon]

@IzhiSu It was the same when I trained the model from scratch. I tested the weight of first best_epoch=~~~~.ckpt in the checkpoints folder and the test results on KITTI pedestrian were 66.192/91.008(Success/Precision).

@slothfulxtx Can you tell us how many GPUs you use for training and which checkpoint you used for testing?

[StrivedTye]

@IzhiSu It was the same when I trained the model from scratch. I tested the weight of first best_epoch=~~~~.ckpt in the checkpoints folder and the test results on KITTI pedestrian were 66.192/91.008(Success/Precision).

@slothfulxtx Can you tell us how many GPUs you use for training and which checkpoint you used for testing?

Hi,

I have the same problem. Did you solve it?

[slothfulxtx]

Hi, sorry to hear that. Based on my memory, I use only 2 3090Ti GPUs for training, under the conda environment provided by the freeze.yml. No more tricks were used in this project because I was not very familiar with DDP training or multi-node training tricks at that time. To save the storage resources for other 3D datasets, I had to remove some non-urgent files a few months ago. So I've lost some experimental details about this project. Sorry.