Skip to content

Implement basic vulkan renderer

Jan Kelling requested to merge github/fork/nyorain/vk into master

Finally had some time to put this together.

Current state

  • working and tested in basic scenarios (displaying terminal, cursor etc) on all backends, all examples (and rootston) work, screenshotting as well
  • buffers from hardware accelerated clients (dmabuf/wl_drm) are not (yet) implemented for the vulkan renderer. Don't know how to test dmabuf and to get mesa egl/vulkan clients to work the vulkan renderer would have to implement wl_drm at the moment; i'm not sure how to do that correctly. Since it was was mentioned that this might not be needed in the future at all, leaving such buffers for another pr. The dmabuf implementation would also profit from the mentioned (and hopefully soon releasing) vulkan dmabuf modifiers extension. See @fooishbar's comments ([1], [2])
  • As the pr says this renderer is fairly basic and some things can be optimized. The pr is rather large like this already, so i'd rather postpone that (see bottom of the pr)
  • We also make some assumptions in the renderer not guaranteed by the spec. Some can be fixed with some work on the renderer, others would probably require large workarounds or changes to wlr interfaces. Those are usually explicitly marked in the code.

You can try it out by starting programs with WLR_RENDERER=vulkan .... Note that besides the vulkan dependency it currently requires glslangValidator (arch package glslang) to compile the shaders from glsl into spirv c headers. This dependency is only required at build time and only to recompile the shaders; we could also include the compiled shader headers in the repo, removing this dependency.

Interface changes

As expected, this has to touch wlr interfaces (only unstable ones, thanks for waiting with stabilizing them). The main changes are in renderer, output and backend interfaces. This also introduces a new abstraction wlr_render_surface, generally is the design inspired and close to the ideas from renderer redesign v5. This allowed me to remove all direct egl and gl usages from backends. egl initialization is still done by the backend and the render_surface creation interface is explicit (using wl/xcb/gbm primitives) so that we don't have any wlr_backend_is_x chains in the renderers. But otherwise renderers must be aware of all backends they support and do so explicitly. Therefore, this moves some code from the backends (mainly drm, drm/renderer.c) to the vulkan/gles2 renderers.

In the renderer redesign issue, @SirCmpwn mentioned that this approach makes it hard to write new backends or renderers since for a new backend the render interface must be extended and support implemented for existent renderers (if possible) and for a new renderer one has to explicitly implement support for the supported backends.

The thing with this is: there is no way without that difficulty. When creating a new renderer, it has to know how to render on the backends its supports. This cannot be done in a general way since e.g. vulkan has explicit methods for creating surfaces and egl on wayland requires creation of a wayland-egl-window and rendering on the drm/gbm backend requires a lot of completely different backend AND renderer dependent primitives. And when creating a new backend, it is either possible to use the primitives the renderer interface already uses (e.g. gbm) or the render interface just has to be extended to be able to pass the new primitives from backend to renderer. We could (what i did first, see some older commits) create a more general create_render_surface interface, where a void* handle is passed to the renderer. But this requires that renderer and backend both know what the handle should be (and casting it into structures containing all information from the backend, see e.g. the gbm case where a single handle isn't enough information) and to if (backend_is_x(backend)) ... chains in the renderer which seemed more limited and generally not like a good idea.

This digs somewhat into the current drm backend implementation to make this work, so it should be tested on as many setups as possible and definitely receive the mandatory @ascent12 review. I realize that this pr is rather large and may require some changes before everyone is happy merging it so please give me the chance to get it into that state with as much reviews and comments as possible. Most of the later commit messages are also rather detailed and outline design decisions as problems came up. I'm not sure how many of you are experienced with vulkan, but some reviews of the vulkan usage would be appreciated as well (although i heavily relied on the debug layers while developing). Otherwise, see the vulkan spec for information.

If you (categorically/at the moment) reject such a pr or a vulkan renderer altogether, please also let me know up front. Although i spent quite some time on this i did so without communicating/asking and mainly because it was a really interesting task. So i'd understand if you chose not to merge this. If you do, i will gladly maintain this renderer in the future. Will also be as often as possible on the irc the next time, just ping me for questions and discussion.

This also fixes some non-vulkan issues though i found on the way so you at least probably want the wlr_output_damage_begin fix and the improved wayland output cursor setting.

Notes

A list of notes/decisions i wrote down while writing the renderer as well as ideas for future improvements and optimizations.

  • with vulkan we can efficiently read from textures but wlr has no interface for this. It is only needed in one place at the moment iirc: in the drm backend when setting the cursor (there it is currently first rendered and then read from the dummy render surface). Since it's called every time the cursor is changed when using the drm backend, this should be a rather important optimization (and i guess we can implement it more efficiently in gles2 as well using a dummy framebuffer we read the texture from instead of rendering the texture first?)
  • vulkan and gles2 gbm render surfaces currently always use 3 buffers statically. that can be improved with explicit fencing (at least for atomic modesetting) and releasing on pageflip again (see drm backend comment) but since this requires more changes to the drm backend/wlr interface and is mainly a optimization thing, it is left for later
  • vk_khr_incremental_present is currently not used (optimization) implemented, buffer age for swapchain surfaces fixed by pre-acquiring an image
  • all textures use currently linear tiling and are created on host visible memory. This has a rather large performance impact and can be improved with some uploading logic staging logic for optimal shm textures now implemented
  • can textures ever be resized? if so this is currently not implemented on the vulkan renderer nope, not allowed
  • we currently just choose the first physical device. A better approach would be to choose them by what they support but also make this configurable. Multi gpu support is currently not implemented in any way and due to the way we interface with the drm backend, the vulkan renderer might not work at all when using the drm backend with multiple gpus.
  • in vulkan gbm render surfaces, we currently create gbm_bo's and import them into vulkan. we could also create vulkan images, export them as dmabufs and import them as gbm_bo's. Might have an impact on performance/portability (?) no, wouldn't be possible to specify gbm_use_scanout this way; can be improved with modifiers extension
  • since the currently packaged mesa version does not implement vk_khr_display this does not implement such a backend. That is a somewhat independent topic anyways. Would like to implement it though in the future (the next mesa version implements it iirc). Started a VkDisplayKHR backend implementation here on top of this pr, roughly working with mesa 18.2. To not further blow this pr up, will submit it as a new one afterwards.
  • the vulkan renderer currently implements a custom swapchain for gbm render surfaces. It might be possible to use gbm_surface there as well but since the documentation (in the gbm implementation, not the header) explicitly mentions eglSwapBuffers and when it must be called, i'm not sure and haven't tried it. nope, gbm_surface does not work with vulkan
  • the current implementation requires vulkan 1.1. If this could be problem on any setup we want to support, let me know. The same can be achieved (with some work and additional extension loading) with vulkan 1.0.

Merge request reports