Skip to content

Initial SLAM tracking for Valve Index

Mateo de Mayo requested to merge mateosss/monado:mateosss/lhslam into main

This MR enables the vive driver to use the SLAM tracker. Tested with a Valve Index and Basalt.

There are still many problems as pointed out in the TODOs from my working branch but it is probably a good idea to upstream this nonetheless.

Some of the key issues are:

  1. Either the factory calibration is wrong or I was unable to do the proper conversion to OpenXR coordinates. As such you will need to manually calibrate your headset with Basalt. As a starting point there is an example calibration you can use.
  2. The v4l2-vive-lighthouse builder architecture is suboptimal, to say the least. I tried to follow how the WMR does it as much as I could, but having the camera stream in the v4l2 driver separate from the vive driver is annoying.
  3. The performance impact of the frame copies that occur in Basalt are now more noticeable because the Index stream bigger images and more frequently compared to WMR.
  4. There is no IMU to middle-of-eyes correction, which is particularly noticeable for the Index.
  5. I suspect sample timestamps are slightly off for some reason because Basalt features flicker quite a bit compared to other devices.

Having said all of this, it works, but keep in mind that it is in an early stage.

Merge request reports