Skip to content

Add SLAM tracking to WMR driver

Mateo de Mayo requested to merge mateosss/monado:mateosss/wmr-slam into main

This MR adds the wmr_source class (inherits from xrt_fs) which should be in charge of managing data sources coming from WMR headsets. Wrapping sources in this xrt_fs allows for controlling the stream of data and the sinks towards which that data is forwarded to. With the data source set, the SLAM tracker is initialized and the video/IMU data is streamed into the SLAM sinks for tracking.

The usability of this integration still needs work.

  • Tracking with Basalt is working, but at frame rate (30fps) thus very clunky.
  • Exposure/gain needs to be manually adjusted by the user before submitting frames to the SLAM system (some work has been done on autoexposure algorithms, but I preferred to leave that out of this MR for now as it needs more fine tuning).
  • It is necessary for the user to recalibrate its headset using Kannala-Brandt4 parameters (i.e., OpenCV fisheye with k1, k2, k3, k4 for each camera).
  • The calibration for Basalt needs to be manually set with a Basalt config file.

I'll be working on improving these items in following MRs.

Edited by Mateo de Mayo

Merge request reports