---
title: "Monado - Positional Tracking with Playstation Move"
layout: main
---
* TOC
{:toc}
# Playstation Move and PSVR 6DoF tracking
Playstation Move and PSVR tracking is a work in progress and the implementation described here will be improved/changed over time.
The camera setup has to be done once per camera and does not have to be repeated for PSVR and Playstation Move controller setup.
The PSVR is fully set up after calibrating the camera. The Playstation Move controller setup requires an additional bluetooth pairing step.
# Requirements
* Bluetooth adapter (for Playstation Move)
* Stereo Camera
* PS4 Camera
* Generic: 3d-1mp02 is supported out of the box, others may need some work
# Camera Setup
## PS4 camera: firmware
By default the ps4 camera does not work as a camera, a firmware file has to be loaded onto the device first.
Download the firmware upload script, and a `firmware.bin` file to the same directory. The `firmware.bin` file used here with md5sum `99ca1e97ea2bb9edaf4c6f511292b4e2` comes with uvc support and has been tested with Monado.
wget https://raw.githubusercontent.com/ps4eye/ps4eye/master/python/ps4eye_init.py
wget -O firmware.bin 'https://github.com/psxdev/luke_firmwares/blob/master/101_85C8E0_64036.bin?raw=true'
The `firmware.bin` file is uploaded with the command
sudo ./ps4eye_init.py
This should print `PS4 camera firmware uploaded and device reset`.
The camera should now work with standard UVC tools like `guvcview`. In Video Controls, choose the first entry called `USB Camera-OV580: USB Camera-OV`.
If the camera does not show up, or guvcview can not produce images, try a different USB port/host controller and especially without a USB hub. The ps4 camera is known to not work on all USB controllers and hubs.
The script has to be run every time the camera is plugged in or the system is rebooted.
To automate loading the firmware when the camera is plugged in, create an udev rule file `/etc/udev/rules.d/91-ps4.rules` with the content
```
ACTION=="add",SUBSYSTEMS=="usb",ATTRS{idProduct}=="0580",ATTRS{idVendor}=="05a9",RUN+="/usr/local/opt/ps4eye/ps4eye_init.py"
````
Adjust the path `/usr/local/opt/ps4eye/ps4eye_init.py` to where you stored the script and the firmware.
## Calibrating the camera with monado
Currently Monado only supports interactive calibration with a checkerboard pattern. In the future a calibration file will be distributed with monado and calibration will be simplified, perhaps even unnecessary.
The `monado-gui` utility is used to calibrate the camera before using it with Monado. This is necessary to compensate for the usual camera/lens distortion. `monado-gui` is usually installed in /user/bin. In a monado build tree the gui utility is located at `./src/xrt/targets/gui/monado-gui`
1. Get a checkerboard grid. It can be printed on a sheet of paper or displayed on a monitor. For example you can use a checkerboard generator:
1. Go to https://calib.io/pages/camera-calibration-pattern-generator
2. Set `Target Type` to `Checkerboard`. The calibration looks for black and white squares with no symbols.
3. Set `Rows` to `7` and `Columns` to `9`. Other configurations can work too, but 7x9 is a good default.
4. Show the resulting image/pdf on screen or print it.
5. Measure the size of a single square.
2. In the `monado-gui` menu choose `Calibrate`
3. Choose the `USB Camera-OV580` camera (the PS4 camera)
4. Choose the second resolution 1748x408 (the full resolution takes quite some CPU power)
5. Confirm the `7 rows`, `9 columns` setting matches your checkerboard
6. set `Checker Size` to the previously measured size of one square **in meters** and click Done.
7. Move the camera around, until it captures enough images.
* tilt the camera and view the checkerboard from different angles
* capture the checkerboard in as much of the camera's field of view as possible
8. After the calibration finishes, do not forget to click **save** before exiting.
Two configuration files should be stored. `~/.config/monado/config_v0.json` contains information about the calibration and `~/.config/monado/PS4.calibration` contains the actual calibration data.
After these steps a PSVR HMD should automatically begin to track when Monado is started.
# Controller setup
## USB/HID Permissions
Even when connected over bluetooth, your user will require the permissions to access the psmove's hid-over-bluetooth device.
If you use the ["xr-hardware"](https://gitlab.freedesktop.org/monado/utilities/xr-hardware) udev rules, you don't need to do anything, it includes rules to set permissions for the psmove controllers.
If you want to install only rules specific to the psmove instead, you can follow [the psmove documentation](https://github.com/thp/psmoveapi/blob/8bcc80d4fec9087fdb2890dc357f1af8f989c318/docs/pairing.rst#L79-L87).
## Bluetooth pairing the controllers
Bluetooth pairing the controllers requires a USB connection. However IMU data can only be read through bluetooth, once the controllers are paired, reading IMU data through USB is not supported.
The pairing procedure is detailed at [https://github.com/thp/psmoveapi/blob/master/docs/pairing.rst](https://github.com/thp/psmoveapi/blob/master/docs/pairing.rst).
In short:
git clone --recursive https://github.com/thp/psmoveapi.git
cd psmoveapi
cmake .
make
sudo ./psmove pair
Pairing doesn't always work reliably, it may be necessary to restart the bluez daemon `sudo systemctl restart bluetooth` and repeat pairing until it works. When powering on the controller with its power button, it should automatically connect to the PC, and bluetooth tools should show the controller ("Motion Controller") as connected. Trying to initiate a connection from the PC usually does not work.
# Controller Usage
The psmove poses and buttons are fully exposed through a monado specific interaction profile `/interaction_profiles/mnd/ball_on_stick_controller` (see bindings [in the code](https://gitlab.freedesktop.org/monado/monado/blob/f3803f7365c5df1e1d0681a8f5bcb736ea93c199/src/xrt/state_trackers/oxr/oxr_binding_data.h#L405-768)), and limited bindings are available through the khronos simple controller interaction profile.
When a PS4 camera with uvc enabled firmware is connected, a calibration file is present, and a paired and connected psmove controller is connected, monado should automatically enable positional tracking for the psmove controllers.
As an example, `hello_xr -g Vulkan` should have the psmove controller enabled as a positionally tracked "khronos simple controller", including the trigger and haptic feedback.
To debug issues, monado implements a debug UI in a separate window which can be enabled with an environment variable: `XRT_DEBUG_GUI=1 monado-service` and then `hello_xr -g Vulkan`. If monado can find and enable the camera, there will be windows `HSV Filter` and `PSMV Tracker #1` and `PSMV Tracker #2` available in the debug UI, showing the raw camera image as well as the color filtered camera images.