Host-SLAM tracking for the RealSense driver
This MR enables the use of a RealSense device that has video and IMU streams to be tracked with the slam tracking implemented in !889 (merged). It was developed around a D455 camera but if you provide the appropriate config files, it should work with any other RealSense device that has video and IMU streams.
About the changes
- This MR adds a new kind of RealSense device that can be tracked with external (host) SLAM.
- The existing device was renamed from "6dof" to "ddev" (short for Device-SLAM tracked device) to differentiate itself from the new "hdev" (Host-SLAM tracked device).
- The necessary probing logic was implemented so that the correct tracking method is used.
- Some strings from the previous
6dof
device have changed:- JSON field from
realsense_config
torealsense_config_ddev
- Device serial from
Intel RealSense 6-DOF
toIntel RealSense Device-SLAM
- JSON field from
- Introduces the options
RS_TRACKING
for selecting the preferred tracking method andRS_HDEV_LOG
for logging. - Introduces the following optional json fields (if not provided, defaults for the D455 will be used):
realsense_config_hdev { bool stereo, // Whether to enable the second camera stream rs2_format video_format, // An integer that will be casted to a rs2_format // (if required I can work to make this an enum instead) int video_width, int video_height, int video_fps, int gyro_fps, int accel_fps, rs2_stream stream_type, // Similar to video_format but with rs2_stream int stream1_index, // More than one stream for a particular device-pipeline // could exist, this allows the user to select which one to use int stream2_index }
Reviewing and testing it out
- Merging any of !893 (closed), !906 (merged) or any fixes to the regression I'm temporarily patching with 140ba7c1, would require an update on this MR.
- It is necessary that someone tests that the T26x functionality is still working (take into account the renames of
realsense_config_ddev
andIntel Realsense Device-SLAM
, besides that no extra changes should be needed). - I'll update this guide to indicate how to make the D455 and any other realsense cameras work with Kimera and Monado.
Other considerations
- One of the last features my prototype branch has that I've not yet implemented into Monado is the ability to record the realsense stream to an Euroc dataset, I might do that in a different MR.
- There are two segfaults that happen rarely (mainly on exit) related to a known Kimera visualizer issue, they seem to go away when you disable it (i.e., set this to
false
).