Test

[LiliMeng]

Hi,

  Thanks a lot for the code!

  How to test it in RGBD benchmark or Microsoft 7Scene dataset? 

  Thanks again!

[connerbrooks]

The current implementation works with TUM data-sets using association files to read the images and poses. You can create the association files as described here.

I have yet to evaluate with 7-Scenes as I've had issues successfully downloading any of the scenes.

[LiliMeng]

Thanks a lot!

It works by changing the bool train = true in the main.cpp file :)

$./relocforests_example /home/lili/BMVC/data/rgbd_dataset_freiburg3_long_office_household/
[Reader] Opening: /home/lili/BMVC/data/rgbd_dataset_freiburg3_long_office_household/associate.txt
[Reader] Read 2487 frames from association file:
/home/lili/BMVC/data/rgbd_dataset_freiburg3_long_office_household/associate.txt.
[Tree 1] Generating Training Data.
[Tree 1] Training.
[Tree 1] Train time: 1302.19 Seconds
[Tree 2] Generating Training Data.
[Tree 2] Training.
[Tree 2] Train time: 659.343 Seconds
[Tree 3] Generating Training Data.
[Tree 3] Training.
[Tree 3] Train time: 1438.73 Seconds
[Tree 4] Generating Training Data.
[Tree 4] Training.
[Tree 4] Train time: 1406.35 Seconds
[Tree 5] Generating Training Data.
[Tree 5] Training.
[Tree 5] Train time: 1570.67 Seconds
Is forest valid:1
Done.

How long does your training procedure usually take?　How many trees do you need for training?

What's the accuracy and speed of your code? Thanks a lot!

[LiliMeng]

It seems that it has pretty large errors for testing the frame201:

Ground truth:
1341847987.790000 0.5454 1.4786 1.5372 0.3095 0.7924 -0.4825 -0.2087 1341847987.795290 depth/1341847987.795290.png 1341847987.794708 rgb/1341847987.794708.png

known pose rotation in Euler Angles: 125.145, -39.003, -158.158
known pose translation: 0.5454, 1.4786, 1.5372

found pose in euler angle: 151.68, -37.0323, 133.105
found pose in translation: -1.30432, -0.814366, 0.681479

How is going with your result? :) Thanks:)

[connerbrooks]

It seems you got it running, sorry about that, there is a fix for the command line interface in the other branches. I'll have a fix in master soon.

Training normally take ~2 minutes per tree, and I use 5 trees per forest. So overall train time take around 10-15 minutes, while evaluation takes about 3 seconds per frame.

From the tests I have run the results are very close to the results from the paper. In the paper depending on the scene they were able to get accuracy from 39.2% to 95.5% (Successful frames being within 5cm translational and 5 degree angular error).

I tested with the Frieburg Desk1 and Desk2 datasets and was able to get nominal accuracy.

[LiliMeng]

Thanks! Which branch did you get the results close to the results from the paper? It seems that you have three branches :)

[connerbrooks]

Each of the branches have the same underlying forest code. The visualization branch adds pose visualization, and pipeline_integration is work to make the code more generic for integration into scanning applications.

[LiliMeng]

Thanks! Does all the three branches have the same accuracy? As it seems that for the frame201 (the 201th frame in the ground truth file of freiburg3_long_office_household, and number 200 in your code), it has very large errors both on rotation and translation. Is it normal?

known pose rotation in Euler Angles: 125.145, -39.003, -158.158
known pose translation: 0.5454, 1.4786, 1.5372

found pose in euler angle: 151.68, -37.0323, 133.105
found pose in translation: -1.30432, -0.814366, 0.681479

[connerbrooks]

Yeah, that's normal. Frame 201 was tracked well in the Freiburg Desk1 dataset.

To test the accuracy of the specific data set you are using you could iterate through all the frames and calculate the error per frame, then calculate the same percentages of tracked frames from the paper.

[LiliMeng]

Is that because the freiburg3_long_office_household has larger scene environment? As the paper's dataset is only tested on small dataset. But the freiburg3_long_office_household has larger environment scale, so it perform badly? Freiburg desk1 and desk2 have relatively small scenes which are similar to the dataset of the paper.

[LiliMeng]

For fr1_desk, the frame201, it got pretty small errors.

Train time: 9.30508 Seconds
found pose:
0.561904 -0.561006 0.607894
-0.826741 -0.356335 0.435344
-0.0276164 -0.747193 -0.664033

33.2491
142.563
-135.046
-0.660553
0.4693
1.43855
known pose:
0.546743 -0.559637 0.62274
-0.837075 -0.34975 0.420428
-0.0174342 -0.751173 -0.659669
32.5106
141.482
-134.327
-0.6776
0.4748
1.4451