diff --git a/gsoc-2023.md b/gsoc-2023.md index 22c2a4f20a91f716e84672754f3e5371dc38ba97..0581f04e6c89cb70cd6b89f20dd577aed575dad8 100644 --- a/gsoc-2023.md +++ b/gsoc-2023.md @@ -97,6 +97,77 @@ Difficulty: medium Project Size : Small (\~150 hours) +## Visual-inertial tracking module expansion + +This is mostly an integration task that will make the visual-inertial tracking +module more versatile and usable for more people. The student will learn a lot +about build systems, dependency management, CMake, C++, etc, for Linux, Windows, +and Android. + +One of the halves of this project is integrating two tracking systems: DM-VIO +and SVO Pro. DM-VIO (GPL) is one of, if not the best monocular-IMU system, even +competing with stereo-camera systems. If we plan to make our tracking work on +phones, this looks like a very good starting point. SVO Pro looks like one of +the SLAM systems with more polishing out there implementing very cool +techniques. I think it has a good chance of competing or even surpassing what we +have currently, and would be nice to have that as a comparison point for our own +system. + +The other half is dedicated to platform support. Currently, our tracking module +only runs on Linux. Basalt is the underlying SLAM system we use and we need it +running on Windows and Android; many Monado users have asked for this. The idea +is to run camera-IMU datasets through the entire monado-basalt pipeline on both +platforms. + +If time remains, and as a stretch goal, expanding and adapting the current +Android driver in Monado to make it feasible to run the just-integrated Basalt +tracking would be awesome! + +Deliverables: +* At least two of the following four points: + * DM-VIO running EuRoC datasets through Monado on Linux + * SVO Pro running EuRoC datasets through Monado on Linux + * Basalt running EuRoC datasets through Monado on Windows + * Basalt running EuRoC datasets through Monado on Android +* Optionally, expansion of the Android driver with more sensor information for visual-inertial tracking. + +Requirements: +* Some C/C++ experience +* Some understanding on how to use multithreading primitives +* Basic understanding of C/C++ build procedures +* Available Linux/Windows/Android devices depending on which points will be addressed. +* Experience in either C++, CMake, pkgconfig, Linux/Windows/Android building is a plus. + +Difficulty: medium + +Project Size : Large (\~350 hours) + +## Passthrough + +Passthrough for VR is an essential feature in current headsets which have +multiple cameras available on them. It lets the VR application project the +video feed from the cameras into meshes so that the user can be still be aware +of its surroundings and interact with them while they are in the VR experience. + +This is also one of the base features needed to be able to have a proper room +setup in Monado. Furthermore, if we manage to do full FoV passthrough, that will +be a key way of observing drift and accuracy in our visual-inertial tracking +systems. The student will learn about OpenXR, Monado, C, C++, and some computer +graphics/vision basics. + +Deliverables: +* Implementation of one of the OpenXR passthrough extension in Monado +* Simple OpenXR application that uses this extension +* A module to stream a video feed for passthrough that is easy to reuse in other drivers +* Stream video either from a webcam or a video file. + +Requirements: +* C/C++ experience +* Linux experience + +Difficulty: medium + +Project Size : Large (\~350 hours) ## Your own idea