Skip to content

Draft: turnip: Initial support for sparse binding and sparse residency

Use the new freedreno kernel uapi in development by @robclark that uses the core DRM memory manager to implement support for sparse binding, and sparse residency for buffers. Sparse residency for images isn't ready yet, due to issues with the specification resulting in test failures because we can't implement sparse images with VK_IMAGE_USAGE_MUTABLE_FORMAT_BIT, but this is enough to fully exercise the new kernel API. This means that games requiring FL12_0 can't run yet, but hopefully soon.

The new uapi completely reinvents how memory allocation works in a way that is tailored for Vulkan. When a GEM object is created, it is not assigned an iova by mesa or the kernel. Instead, mesa submits bind commands to a sparse bind queue to bind a given range of pages in the object to a given iova. Internally, we create a driver sparse queue for driver-initiated binds and unbinds, and synchronously map the entire range of the BO to a driver-assigned iova after creating it and unmap it before destroying it. The Vulkan sparse queue exposed to apps is a second sparse queue from the kernel's PoV. Userspace is in complete control of the GPU address space, and unmapping does not block on GPU work. This means that the BO list and "zombie" BOs are all unnecessary, and submitting actual GPU work now takes a simple list of iovas instead of BO + offset. As an extra bonus, there is a concept of non-shareable GEM BOs to match the concept in turnip, which allows lower kernel overhead on submit. We always use the new uapi when available, because it's a much better match for us.

This MR also includes support for sparse on kgsl, using the "virtual BO" uapi there. This is very different from how the new freedreno uapi works, and much more restrictive, so we have to bridge the difference in tu_knl. The shared abstraction looks a lot like kgsl, which is very close to how Vulkan works.

There are still a few loose ends that depend on kernel support to make sure the new path matches the old path in features:

  • Support for marking memory to dump. This currently uses a flag in the BO list which is going away. This means that devcoredump and recording traces via /sys/kernel/debug/dri/0/rd won't work currently. We need to come up with a new mechanism.
  • Support for debug names for BOs. Maybe we can just enable this?

The kernel side can be found at drm/msm!144 or https://patchwork.freedesktop.org/series/142263/

This MR depends on !32165 (merged) in order to make supporting sparse bind submits easier.

Edited by Connor Abbott

Merge request reports

Loading