Proposal: Clarify buffer handling with PW_*_MAP_BUFFERS especially with respect to dmabuf
When a client sets the PW_*_MAP_BUFFERS, pipewire will mmap the filedescriptors for MemFd and DmaBuf buffer types. This has some problematic implications for negotiation and buffer handling.
TLDR: Stop mapping dmabufs inside pipewire and extend clients announced buffer data_Type capabilities with pipewires capabilities.
Currently pipewire uses mmap for all types of buffer mapping. This has some problems for dmabufs
- mmap only supports linear memory aka. modifier 0. (I1)
- mmap does not respect the implicit fence used by dmabufs and requires additional ioctl. (I2)
- downloading buffers from the gpu to main memory is usually slow. Consumer should only request dmabuf buffer types if they want to process those buffers further on the gpu, e.g. using vaapi, or are using vulkan, opengl, etc. for downloading. If a consumer is only able to process buffers in main memory it's in most cases better to request buffers from main memory and let the producer download the buffer, since it has usually better tools for it. (I3)
- Since clients are not aware of the current mapping capabilities of pipewire, there is currently no good way of setting parameters like SPA_PARAM_BUFFERS_dataType and SPA_FORMAT_VIDEO_modifier. Pipewire should take over setting the params for the client. E.g. a client only uses the MemPtr mapped onto spa_buffer buffer->datas->data and declares capabilities for MemPtr with PW_*_MAP_BUFFERS. Pipewire should declare capabilities for all supported buffer types (currently MemPtr, MemFd and DmaBuf with modifier 0). (II1)
- Clients can implement additional buffer handling capabilities, which can surpass the capabilities of pipewire for a specific buffertype, which would make merging those parameters complicated. E.g. a client wants to rely on pipewire for MemPtr and MemFd handling, but wants to support DmaBuf with arbitrary modifiers. (II2)
- (I1) & (II2): Using PW_*_MAP_BUFFERS with a custom DmaBuf handling works for consumers, but creates an failed to map buffer error from pipewire.
Remove dmabuf handling from mmap buffers:
Stop offering mapping of dmabufs for clients (see !324 (merged)). This would avoid most problems regarding dmabufs without adding dependencies. Together with a mechanism to add the MemFd buffer dataType to a consumer using PW_*_MAP_BUFFERS with MemPtr, this would improve clarity of a consumers capabilities for all participants (Not setting any capabilities would still result in no value set). (S1)
Omit mapping dmabufs with modifier not 0:
This would solve the occurring mapping errors, but can lead on it's own to unexpected behaviour, if a client is not signalled about it. The client and pipewire would need a mechanism to either continue, when the client is able to handle dmabuf on it's own, or renegotiate the buffer type. (S2)
Overwrite client params precisely with pipewire capabilities:
This would solve the negotiation issue, since all needed informations are present, but would hinder clients in implementing custom (and extended) capabilities. On the dmabuf side, this would not improve dmabuf handling and prevent consumers from using dmabufs with arbitrary modifiers. (S3)
Implement S1 to create a simple and clear and therefore reliable ruleset to negotiate capabilities, without adding dependencies or much overhead to pipewire or restrict clients. Mapping dmabufs is a complicated topic, which needs apropriate tools, which is better suited for an implementation as a separate client, which is not restricted in dependencies and could offer additional buffer transformation capabilities, since common used formats may differ for buffer types. PW_*_MAP_BUFFERS should only handle all dataType cases, where clients can assume full support for the MemPtr api.
Edit: Add thought about MemPtr api