PIXMAN_FILTER_SEPARABLE_CONVOLUTION yields incorrect results on big endian
In the function bits_image_fetch_separable_convolution_affine
in pixman-fast-path.c
, components are extraced from uint32_t
s using RED_8
et al, which always extracts bits with the same weigths regardless of endianness, for example the red component is always associated with the bit weights 65536 through 8388608 (2^16 through 2^23). This is consistent with how Cairo stores pixels, and as far as I can understand also with how pixman wants things internally.
However, when writing back the result to buffer[k]
, then there is suddenly an #ifdef WORDS_BIGENDIAN
which instead wants to associate the red component with bit weights 256 through 32768 (2^8 through 2^15) on big endian. This is clearly inconsistent and causes the pixels to become reversed. Removing the #ifdef WORDS_BIGENDIAN
and always using the assignment with the correct bit weights makes PIXMAN_FILTER_SEPARABLE_CONVOLUTION work correctly (just like PIXMAN_FILTER_BILINEAR does) on big endian.