AMDGPU: Solution to reduce the size of ASIC headers in the kernel repository

Motivation

The apparent size (du -hsc *) of the drivers/gpu/drm/amd/include directory is 32.7% of the whole git tree (excluding .git), and it's not going to stop growing. The directory only contains hw register definitions ("ASIC headers") that are mostly unused.

This presents a solution to:

• reduce the source code size of those ASIC headers by 98% (estimate)
• keep it low in the future
• allow developers to use all register definitions despite that

Solution

1. Move ASIC headers to a separate repository. The kernel tree will only contain necessary definitions. (see step 2 for how that's achieved)
2. If more definitions are needed:
   • Copy the header files that you need from the ASIC header repository into drivers/gpu/drm/amd/include.
   • Make driver changes.
   • Run a python script that will trim all header files in drivers/gpu/drm/amd/include and keep only the definitions that are used by the source code. (the python script can be in the kernel repository or in the ASIC header repository)
   • Make a commit with the trimmed definitions.

How the python script will work is up to the implementer.

External projects that use the ASIC headers (Mesa, AMD tools) will switch to using the ASIC header repository instead of the kernel repository.

This is not a thread about Plymouth or slow boot times.

added AMDgpu amdkfd labels

changed the description

The headers are the official "upstream" source of the IP information in the public. Any "new" repo would have to be maintained by someone with authority to say what is and isn't public. Otherwise projects like umr will be compromised.

Literally nothing changes in terms of "IP clean" expectations.

There is something really similar that is going on for the X86 CPUID expected output data documentation: https://gitlab.com/x86-cpuid.org/x86-cpuid-db

What's going to be happening is that is the centralized database, and they will be generating a compatible format to be periodically updated to be included in the kernel.

Sounds like a plan to me. Linus and other upstream Linux maintainers have been complaining about that for years.

Is this really a a problem is practice? Header files compress pretty well and then this adds a bunch of extra steps when we need to add new registers or bitfields. I'm worried people will manually update the in kernel ones when they need to add something and then forget to update the official out of tree ones.

See the following from Phoronix: https://www.phoronix.com/news/AMDGPU-Driver-Plymouth-Woes

The register headers have no impact on the size of the driver binary. The headers only affect the size of the source code.

The register headers have no impact on the size of the driver binary.

Yeah AFAICT that real issue is that compiled binary is larger due to a lot of duplication of C source code across IP versions and because of the DC code.

The systems pointed out in the bugs in that article are slower/older systems, so loading a big driver binary and decompressing it will take longer. Modern hardware can do this in at least 1/3 or faster of the times quoted there.

As distros migrate to signed UKIs I think it's worth considering moving amdgpu out of the initramfs portion of the UKI and either into the vmlinuz portion of the UKI or just loading it from the rootfs. The former will fix this performance problem on older systems and have maximum compatibility and best experience.

Also; regarding that article setting up Plymouth to use simpledrm by default will help. That's being discussed for the next version:

plymouth/plymouth#264

The register headers have no impact on the size of the driver binary.

Mhm, but they generate a lot of extra debugging symbols.

Is this really a a problem is practice?

That's up to you and Christian to decide.

• Run a python script that will trim all header files in drivers/gpu/drm/amd/include and keep only the definitions that are used by the source code. (the python script can be in the kernel repository or in the ASIC header repository)

Due to how amdgpu uses the headers with macros I think it would be bit difficult to do this purely in python with the source directory and regular expression matching. You would probably need to make a first pass with a pre-processor to expand all the macros and then scan the source tree.

pycpython has a few suggestion how to invoke a C preprocessor. Or parsing AMDGPU macros could be hardcoded in the python script. There is only a few macros I think. It should be easy.

changed the description

Bloated header files have little effect on the quality of the final code and its optimization, but header compression can speed up compilation. The problem with long loading of the driver and Plymouth can be solved by dividing the driver into several smaller drivers by GPU generations, for example RDNA4, RDNA3, etc. Two years ago I looked at the amdgpu driver code, and even in modern GPU architectures, there was a lot of copy-paste code, and there was no code reuse, often there were pieces of dead code copied from the old ATI code. There was also very heavy code in several functions, it seems it was in the driver initialization functions, which the clang-based static analyzer complained about could not analyze and optimize the code properly, and there would be a decrease in performance.