crashes during subtests are unreported, as are all subsequent subtests in the same test
Submitted by Daniel Kurtz
Assigned to Eric Anholt @anholt
Description
I happened to be watching the screen during a recent piglit run and saw a stacktrace fly by. However, when the test run completed, I did not have any "crashes" in my summary.
It turns out that the crash was during the "spec/!OpenGL 1.0/gl-1.0-beginend-coverage/glPixelMapfv" - the glPixelMapfv subtest of gl-1.0-beginend-coverage.
The crash terminated gl-1.0-beginend-coverage. However piglit processes each (completed) subtest as a separate test. Each of the subtests that passed gets the result "pass", with their own identical 'info', 'command', 'time', 'subtest' and 'returncode' fields. Note that the "returncode" field in this case is -6 for all of them (all of the passing subtests), since the master test process crashed. However, there is no separate test item for the master test, to mark the "crash" result. Similarly, since piglit really has no way of knowing how many subtests the master test should run, it doesn't even know which subtest crashed, nor which subtests didn't even get run.