We did not take the sample size into account. Rearrange the tests to have more
conversion test and an extra test case for passthrough operations.

Fixes #759890