nvk: A new Vulkan driver for Nvidia hardware

NVK: A new Vulkan driver for Nvidia hardware

This MR adds a new Vulkan driver for Nvidia hardware. This has been in the works since last May if the git logs are to be believed. For those interested in the development progress of the driver and a bit of why we think now is the time to make NVK happen, I've written a couple blog posts on the topic:

• https://www.collabora.com/news-and-blog/news-and-events/introducing-nvk.html
• https://www.collabora.com/news-and-blog/news-and-events/nvk-update-enabling-new-extensions-conformance-status-more.html

Who wrote it?

While the majority of the code blames to me (@gfxstrand), @karolherbst laid a lot of the ground work and has done a lot of debugging and figuring out what we need to do on various generations. @airlied also helped out quite a bit and has been working behind the scenes doing the kernel work we need in preparation for the new driver.

Additional thanks goes to, in no particular order:

• @DodoGTA
• @anholt
• @georgeouzou
• @mhenning
• @marysaka
• @issor.oruam
• @mohamexiety
• @rmckeever
• @phomes
• @YusufKhan-gamedev
• @benjaminl

Feature support:

I won't claim parity with RADV but we've got a pretty solid set at this point. My GSoC student (@mohamexiety) is most of the way through YCbCr and, once that's done, we'll be able to claim Vulkan 1.2 with reasonable confidence. We've also got a decent set of features on top of that. Not everything required for DXVK, VKD3D, and Zink, but we've got most of it. What remains is either annoying (lines) or is compiler heavy enough that I'd rather just get the new back-end compiler up and going than try to fix all the codegen bugs.

CTS status:

I did a full CTS run right before making this MR:

Pass: 380616, Fail: 1944, Crash: 16, Skip: 1750664, Flake: 130

As you can see, things are in reasonably good shape. We probably aren't going to be claiming conformance for a while yet as I'm still off writing a new back-end compiler, but it's good enough that we've been able to accurately regression test for almost a year.

As per usual mesa protocol, it currently prints a non-conformant driver warning on physical device enumeration.

Kernel requirements:

NVK, when merged, will have a hard requirement on a new nouveau kernel API for buffer binding and submit. There are a couple reasons for this:

1. The old nouveau kernel API has hard-coded limits on the number of BOs supported and the number of pushbufs per exec. This doesn't play nice with Vulkan. For pushbufs, we could split command buffers into multiple execs if needed to work around the limitation but it's annoying. For BOs, though, there's nothing we can do and the limit is way lower than practical if we want to run DXVK on this thing. Even simple DXVK apps blow through the limit pretty quickly.

2. Nvidia hardware requires a few additional bits be programmed in the page tables for depth/stencil attachments (all HW) and color attachments (pre-Turing). This was never a problem for the nouveau GL driver because it does a BO per texture and passes the information through at BO create time. For Vulkan, though, we need to support sub-allocation and that just isn't possible with the current API. We have a hack right now that pulls images that need extra PTE bits out into a BO per-VkImage but that will never be a conformant implementation because it can't support aliasing.

With those two things in mind, NVK will never be conformant on top of the old nouveau UAPI and we need a new one. @airlied and a few other Red Hat folks have been cooking up a new API for us and it's just about ready. The plan is that NVK will never support the old UAPI in any official Mesa release. Even if the code still exists when this MR is pulled in, it will get ripped out before we branch for the 23.3 release. Yes, I'm aware that this may not be the most popular move up-front as users will likely complain about having to get new kernels but they'll need new kernels with GSP anyway if they want anything resembling performance and that will be landing after the new UAPI.

Kernel patches and the NVK MR for the new API can be found here:

• https://patchwork.freedesktop.org/series/112994/
• nouveau/mesa!150 (merged)

At this point, I'm quite happy with the new API and will probably RB the next round of kernel patches.

Plans for merging:

I plan to merge NVK into mesa/main basically as soon as the kernel patches have been merged. The moment I get the okay from @airlied that they're in drm-next, I'll merge nouveau/mesa!150 (merged) into the nvk/main branch and assign this MR to Marge for merging into mesa/main. As far as I know, this branch builds at every commit and doesn't commit any massive meson sins.