Skip to content

Restore no-SLAM, fix replica import, CUDA 11.7, etc#12

Open
jrpowers wants to merge 12 commits intoToniRV:masterfrom
jrpowers:master
Open

Restore no-SLAM, fix replica import, CUDA 11.7, etc#12
jrpowers wants to merge 12 commits intoToniRV:masterfrom
jrpowers:master

Conversation

@jrpowers
Copy link
Contributor

@jrpowers jrpowers commented Jan 1, 2023

This PR addresses a few things I ran into trying to test NeRF-SLAM:

  • I could not run out of the box on Ubuntu 22.04 LTS with CUDA 11.3, so I updated to CUDA 11.7 (the first version with Ubuntu 22.04 support)
  • Then I also had to bump Pytorch on the way, since older Pytorch wasn't available with CUDA 11.7
  • I had trouble building the customized gtsam, so I found a smaller set of edits that I could compile
  • Even after that, I found some gtsam APIs had changed, esp. regarding Quaternions and identity(). I am not sure how it was working for @ToniRV since I didn't spot changes in the gtsam fork that would have addressed it. So I updated a few lines to compile.
  • I tried to run without --slam and found it wasn't working, so I fixed a few things so that could work.
  • I noticed some case-sensitivity issues when unzipping datasets, and noticed the Replica sample ZIP had apparently been renamed too.
  • I temporarily pointed gtsam to my fork for the purposes of reviewing the PR, and I also made a PR on the upstream gtsam with just 2 lines which would make it compatible with NeRF-SLAM (at least the parts I ran).

@jrpowers
Copy link
Contributor Author

jrpowers commented Jan 1, 2023

Looks like the gtsam guys may accept the minimal PR, then we could update to use the plain develop branch. borglab/gtsam#1361

@ToniRV
Copy link
Owner

ToniRV commented Jan 2, 2023

Awesome! Thanks!

@ZZy129326999
Copy link

ZZy129326999 commented Jan 26, 2023

I'm worried.
AttributeError: module 'droid_backends' has no attribute 'reduced_camera_matrix'

H, v, Q, E, w = droid_backends.reduced_camera_matrix(
    self.cam0_T_world,
    self.world_T_body, # TODO(remove, unnecessary, given the math)
    self.cam0_idepths,
    self.cam0_intrinsics[0],
    self.cam0_T_body,
    self.cam0_idepths_sensed,
    gru_estimated_flow, 
    gru_estimated_flow_weight, # TODO: we should pass as well previous pose covariances!, so to not lose information when optimizing
    damping,
    ii, jj, kf0, kf1)

@kadirkun
Copy link

Hi guys, I encountered with the same issue. I was able to come up with the same solution without looking at this PR.
Now I see AttributeError: module 'droid_backends' has no attribute 'reduced_camera_matrix' error as @ZZy129326999.

Is there any reason why this PR is not merged? @ToniRV
@ZZy129326999 were you able to solve this camera matrix issue?

@kadirkun
Copy link

Hi guys, I encountered with the same issue. I was able to come up with the same solution without looking at this PR. Now I see AttributeError: module 'droid_backends' has no attribute 'reduced_camera_matrix' error as @ZZy129326999.

Is there any reason why this PR is not merged? @ToniRV @ZZy129326999 were you able to solve this camera matrix issue?

I solved the reduced_camera_matrix error by replacing the droid-slam's src folder with the src folder provided by the nerf-slam repo. After recompiling droid-slam, the droid_backends module had the reduced_camera_matrix function.

@zhangshuoneu
Copy link

wonderful pr, thanks very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants