libweston: Analyze transform matrices for filter and plane tests

Use some simple heuristics to check if a matrix used for transformations has a result that can be placed on a plane or requires bilinear filtering.