Add initial hand tracking for WMR devices
This MR allows WMR headset cameras to be used for hand tracking. It is still in very early stages. To try it out you can follow this file instructions.
EDIT: For the best experience, put the headset in your head, the model works best from that perspective.
A couple of comments:
-
When integrating the hand tracker I noticed its interface is fairly different from other trackers like
t_tracker_{psmv,psvr,slam}
. Essentially these trackers are glorified sinks that device drivers just stream data to. I also noticed there is a stubt_tracker_hand
that has a more similar interface to what I was expecting but it is not being used. I applied small changes with this in mind like theown_xfctx
option or returning a sink onht_device_create_wmr
. -
We are using OpenCV's camera model which is slightly different from the one on Azure-Kinect-Sensor-SDK (which is probably similar to what WMR is using). In practice, these differences shouldn't be significant as detailed in
wmr_hmd_create_stereo_camera_calib
's documentation.