Run CI tests in CI

It turns out that tests are a lot better when you run them. Embarrassingly, we haven't been running the tests for most of CI in CI for like three months now.

There are a couple of fdno expectations updates in here - one for a persistently flaky trace, and another for a test which changed status whilst we weren't looking - plus a couple of small cleanup commits. The bulk of it is fixing the tests which had regressed whilst we weren't testing the tests, then reworking our pytest setup so it covers everything.