Accurate or normalized depth value in training?

[Renatusphere]

Thanks for your contribution in FusionDepth!

I wonder whether the "depth" in the trainer.py is normalized or the accurate value of the depth in each pixel. When I print them in the training process, the value looks like normalized, which is generally below 1. But in the reprojection procedure, this depth value is used directly.

What's more, when calculating the loss, the depth is multiplied by 26. I have no idea what the "26" means.

[fengziyue]

Hi @Renatusphere :

The depth in the trainer.py is normalized and does not reflect the absolute scale. However, the pose net and the depth net are sharing the same scale so they could be used directly for re-projection.

We observe that the ratio between the predicted depth scale and absolute scale is around 26, so we use it here to roughly filter out the invalid depth points.

[Renatusphere]

Thanks for your reply. I get it!

[GANWANSHUI]

Hi @Renatusphere :

The depth in the trainer.py is normalized and does not reflect the absolute scale. However, the pose net and the depth net are sharing the same scale so they could be used directly for re-projection.

We observe that the ratio between the predicted depth scale and absolute scale is around 26, so we use it here to roughly filter out the invalid depth points.

Hi, thanks a lot for the work. If I want to use the si_loss in other datasets, should I calculate the ratio again? and how to calculate the ratio between the predicted depth scale and absolute scale? Thanks again!

[fengziyue]

Hi @GANWANSHUI :

Thank you for your interest. In the KITTI dataset, the valid depth range is 0-80m, so we roughly scale the predicted depth to the real-world metric and filter out the depth points that are out of this range.

For other datasets, you do not have to calculate the ratio, using any method to filter out the unreasonable depth points should be good. By the way, torch.median(depth_gt) / torch.median(depth_pred) in this line is the ratio between the predicted depth and GT depth.