Skip to content

Host-SLAM tracking for the RealSense driver

Mateo de Mayo requested to merge mateosss/monado:mateosss/d455 into master

This MR enables the use of a RealSense device that has video and IMU streams to be tracked with the slam tracking implemented in !889 (merged). It was developed around a D455 camera but if you provide the appropriate config files, it should work with any other RealSense device that has video and IMU streams.

About the changes

  • This MR adds a new kind of RealSense device that can be tracked with external (host) SLAM.
  • The existing device was renamed from "6dof" to "ddev" (short for Device-SLAM tracked device) to differentiate itself from the new "hdev" (Host-SLAM tracked device).
  • The necessary probing logic was implemented so that the correct tracking method is used.
  • Some strings from the previous 6dof device have changed:
    • JSON field from realsense_config to realsense_config_ddev
    • Device serial from Intel RealSense 6-DOF to Intel RealSense Device-SLAM
  • Introduces the options RS_TRACKING for selecting the preferred tracking method and RS_HDEV_LOG for logging.
  • Introduces the following optional json fields (if not provided, defaults for the D455 will be used):
    realsense_config_hdev {
      bool stereo, // Whether to enable the second camera stream
      rs2_format video_format, // An integer that will be casted to a rs2_format
                               // (if required I can work to make this an enum instead)
      int video_width,
      int video_height,
      int video_fps,
      int gyro_fps,
      int accel_fps,
      rs2_stream stream_type, // Similar to video_format but with rs2_stream
      int stream1_index, // More than one stream for a particular device-pipeline
                         // could exist, this allows the user to select which one to use
      int stream2_index
    }

Reviewing and testing it out

  • Merging any of !893 (closed), !906 (merged) or any fixes to the regression I'm temporarily patching with 140ba7c1, would require an update on this MR.
  • It is necessary that someone tests that the T26x functionality is still working (take into account the renames of realsense_config_ddev and Intel Realsense Device-SLAM, besides that no extra changes should be needed).
  • I'll update this guide to indicate how to make the D455 and any other realsense cameras work with Kimera and Monado.

Other considerations

  • One of the last features my prototype branch has that I've not yet implemented into Monado is the ability to record the realsense stream to an Euroc dataset, I might do that in a different MR.
  • There are two segfaults that happen rarely (mainly on exit) related to a known Kimera visualizer issue, they seem to go away when you disable it (i.e., set this to false).
Edited by Mateo de Mayo

Merge request reports