Skip to content

connection: Ensure buffer sizes do not exceed INT_MAX or PTRDIFF_MAX

Demi Marie Obenour requested to merge DemiMarie/wayland:limit-buffer-sizes into main

Pointer arithmetic beyond PTRDIFF_MAX is broken, so buffer sizes exceeding PTRDIFF_MAX (which is half of the address space!) are a bad idea. Furthermore, the code uses int for sizes in various places, so buffer sizes exceeding INT_MAX are also a bad idea. Therefore, limit buffer sizes to (PTRDIFF_MAX / 2) + 1 or (INT_MAX / 2) + 1, whichever is smaller.

Tests would require 2GiB of RAM and so have been omitted. The test would check for wl_connection_flush() flushing more than INT_MAX bytes in one call, causing it to return a negative number and causing its caller to wrongly believe an error occurred.

The last commit is the actual fix. The first two are fixes for undefined behavior and the remaining are either changes the fix depends on or an optmization. The code should be reviewed commit by commit.

Merge request reports