Skip to content
  • Daniel Vetter's avatar
    framework: Add support for a test timeout · ab2eb660
    Daniel Vetter authored
    
    
    i-g-t tests can take a long time, and for a bunch of reasons (bad
    tuning on disparate set of platforms, stuck in the kernel which often
    can be recovered by interrupting with a signal, ...) that sometimes
    extends to a good approximation of forever.
    
    Hence this adds a per-test timeout value and a bit of infrastructure
    to adjust the results. Test results adjusting is done after calling
    interpretResult so that we don't have to replicate this all over the
    place. This might need to be adjusted for the piglit-native subtest
    stuff, but otoh igt is a bit special with it's crazy long-running
    tests. So I think we can fix this once it's actually needed.
    
    The default timeout is None, so this is purely opt-in.
    
    Note on the implementation: SIG_ALARM doesn't exists on Windows and
    stackoverflow overwhelmingly recommended to go with this thread-based
    approach here. But only tested on my Linux box here.
    
    I've also timed quick.tests run a bit and the overhead due to the
    additional thread we launch seems to be in the wash. So I didn't opt
    to make the thread launching optional if there's no timeout limit.
    
    v2: Also add all the boilerplate needed to handle the new test status
    in summaries. For the color I've opted for two shades of light blue,
    they nicely stick out from the current selection.
    
    v3: Fix GLSLParserTest and ShaderTest. They both derive from
    PlainExecTest but only call the __init__ function of the Test
    baseclass. I haven't really figured why this is and also not really
    what command I should pass to PlainExecTest.__init__, so just
    replicated the init code for now.
    
    v4: Initialize timeout earlier for tests where we use the ENOENT
    handling to skip them.
    
    Fix up the out, err passing from the thread. Apparently my idea of how
    python closures work was completely misguided - like with a function
    call a closure captures a copy of of a reference pointing at the
    original object. So assinging anything to them won't have any effect
    outside of the lambda. Hence we need to jump through hoops and pass an
    array around. Nicer fix would be to create a class or something, but
    that seems like overkill.
    
    v5: Fixup the fraction setting in status.py
    
    Reviewed-by: default avatarDylan Baker <baker.dylan.c@gmail.com>
    Cc: Dylan Baker <baker.dylan.c@gmail.com>
    Signed-off-by: default avatarDaniel Vetter <daniel.vetter@ffwll.ch>
    ab2eb660