From 72a11c46d38a56ce1bbd2140f84bd658e8841346 Mon Sep 17 00:00:00 2001 From: Mateo de Mayo <mateo.demayo@collabora.com> Date: Thu, 9 Feb 2023 11:17:44 -0300 Subject: [PATCH] gsoc: Add SLAM extension and passthrough project ideas for GSoC 2023 --- gsoc-2023.md | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/gsoc-2023.md b/gsoc-2023.md index 22c2a4f..0581f04 100644 --- a/gsoc-2023.md +++ b/gsoc-2023.md @@ -97,6 +97,77 @@ Difficulty: medium Project Size : Small (\~150 hours) +## Visual-inertial tracking module expansion + +This is mostly an integration task that will make the visual-inertial tracking +module more versatile and usable for more people. The student will learn a lot +about build systems, dependency management, CMake, C++, etc, for Linux, Windows, +and Android. + +One of the halves of this project is integrating two tracking systems: DM-VIO +and SVO Pro. DM-VIO (GPL) is one of, if not the best monocular-IMU system, even +competing with stereo-camera systems. If we plan to make our tracking work on +phones, this looks like a very good starting point. SVO Pro looks like one of +the SLAM systems with more polishing out there implementing very cool +techniques. I think it has a good chance of competing or even surpassing what we +have currently, and would be nice to have that as a comparison point for our own +system. + +The other half is dedicated to platform support. Currently, our tracking module +only runs on Linux. Basalt is the underlying SLAM system we use and we need it +running on Windows and Android; many Monado users have asked for this. The idea +is to run camera-IMU datasets through the entire monado-basalt pipeline on both +platforms. + +If time remains, and as a stretch goal, expanding and adapting the current +Android driver in Monado to make it feasible to run the just-integrated Basalt +tracking would be awesome! + +Deliverables: +* At least two of the following four points: + * DM-VIO running EuRoC datasets through Monado on Linux + * SVO Pro running EuRoC datasets through Monado on Linux + * Basalt running EuRoC datasets through Monado on Windows + * Basalt running EuRoC datasets through Monado on Android +* Optionally, expansion of the Android driver with more sensor information for visual-inertial tracking. + +Requirements: +* Some C/C++ experience +* Some understanding on how to use multithreading primitives +* Basic understanding of C/C++ build procedures +* Available Linux/Windows/Android devices depending on which points will be addressed. +* Experience in either C++, CMake, pkgconfig, Linux/Windows/Android building is a plus. + +Difficulty: medium + +Project Size : Large (\~350 hours) + +## Passthrough + +Passthrough for VR is an essential feature in current headsets which have +multiple cameras available on them. It lets the VR application project the +video feed from the cameras into meshes so that the user can be still be aware +of its surroundings and interact with them while they are in the VR experience. + +This is also one of the base features needed to be able to have a proper room +setup in Monado. Furthermore, if we manage to do full FoV passthrough, that will +be a key way of observing drift and accuracy in our visual-inertial tracking +systems. The student will learn about OpenXR, Monado, C, C++, and some computer +graphics/vision basics. + +Deliverables: +* Implementation of one of the OpenXR passthrough extension in Monado +* Simple OpenXR application that uses this extension +* A module to stream a video feed for passthrough that is easy to reuse in other drivers +* Stream video either from a webcam or a video file. + +Requirements: +* C/C++ experience +* Linux experience + +Difficulty: medium + +Project Size : Large (\~350 hours) ## Your own idea -- GitLab