Fix compositing rotation

Prior to this, the mach64 driver did not pass the tscoords (t_tsrccoords2.c) test in rendercheck. This was due to the fact that the driver determined the source image transformations based on only two vectors to the start and end corners of the source image. This fails because with only these two vertices, a 90 or 180 degree rotation and a flip or series of flips are indistinguishable and both satisfy the geometrical constraints.

This makes it so that a transformed vector is computed for every vertex of the rectangle, so that the correct coordinates are sent to rasterization to allow it to rotate the rectangle. I have tested these changes in rendercheck, and they eliminated the failure in the tscoords test while introducing no additional regressions.

I apologize if I have made any mistakes in regards to style, function, or the submission of this request. If there is anything I can do to improve in the future, please let me know.

@alanc I am pinging you because you seemed like the person who has worked on this driver the most recently. I apologize if this is inappropriate.

Thank you for looking over this and for any reply.