RandR matrix computations lose too much precision in fixed point;
computations using the inverted matrix can be as much as 10 pixels off.
Convert them to double precision values and pass those around. These API
changes are fairly heavyweight; the official Render interface remains fixed
point, so the fixed point matrix comes along for the ride everywhere.