pan/midgard: misc cleanups
Here's a few cleanups that are inspired by some Coverity issues.
In particular, Coverity complains that we're needlessly passing large structs per value, and this avoids doing that in most (if not all) cases.
But I decided to do some other cleanups on the way as well.