1. 22 Sep, 2015 1 commit
    • Dylan Baker's avatar
      framework: replace TestResult dict with an object · b365367f
      Dylan Baker authored
      This is a very invasive patch, because it replaces one of our core
      data-structures with a completely different kind of object. This new
      object is not a dict-like object (it doesn't use obj[key] = value
      syntax), instead it's a standard object with the standard object
      attributes. So result['time'] becomes result.time.
      This approach has a couple of advantages. First, it allows us to use
      properties, which allows us to encapsulate a lot of distributed logic
      from the summary module in the results module, and in a way that is
      easier to test for correctness. The second advantage of that
      encapsulation is that correct behavior is used everywhere, instead of
      just in most places. Finally, it allows us to use the flyweight pattern
      on the results objects, limiting the amount of memory consumed.
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
  2. 05 May, 2015 1 commit
    • Dylan Baker's avatar
      framework: add incomplete status. · e62cfdac
      Dylan Baker authored
      This will be used later in this series
      This status has a background color of purple in the html summary, and is
      considered the worst status.
      My logic for making it the worst status is as follows: basically we
      always want this to show up as a regression, because it's a very unusual
      status (only available when piglit is aborted or when the whole system
      goes down).
      v2: - use white text instead of black, it makes things infinitely more
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      Reviewed-by: Mark Janes's avatarMark Janes <mark.a.janes@intel.com>
  3. 03 Sep, 2014 1 commit
  4. 11 Jun, 2014 1 commit
  5. 19 May, 2014 1 commit
  6. 28 Apr, 2014 1 commit
  7. 24 Apr, 2014 1 commit
    • Dylan Baker's avatar
      tests/xts.py: Add a tests file for the X Test suite. · e1057c4a
      Dylan Baker authored
      This adds support for running XTS from piglit.
      This patch was original written by Eric Anholt, but I've cleaned it up
      and reworked parts of it for upstreaming, and as a result I've attached
      my name as the author so ``git bisect'' and ``git blame'' will send
      angry mail my way instead of Eric's.
      v2: - Don't break dmesg reporting
          - Use str.format consistently
          - populate profile after check for xtest symlink
      v3: - rebase against master
          - fix spelling errors
      Signed-off-by: Dylan Baker's avatarDylan Baker <baker.dylan.c@gmail.com>
      Reviewed-by: Eric Anholt's avatarEric Anholt <eric@anholt.net>
  8. 23 Apr, 2014 1 commit
  9. 18 Apr, 2014 2 commits
  10. 23 Jan, 2014 1 commit
  11. 15 Nov, 2013 1 commit
  12. 14 Nov, 2013 2 commits
    • Dylan Baker's avatar
      summary.py: Treat subtests as groups · e0d4bdba
      Dylan Baker authored
      This patch causes tests with subtests to be treated as a group, rather
      than as a test. This means that the status the test itself stores will
      be overwritten by those in the subtest.
      There is one oddity about this to be aware of; a test with subtests that
      crashes or fails before any of the subtests run will report a fraction
      of 0/1 with the appropriate color, even though all of the subtests will
      report Not Run.
      v2: - Add subtests to the results file as full tests (the internal view
            of the json), without this they will not appear in changes, fixes,
          - Render the background color of Not Run tests correctly in HTML
          - Apply subtest fractions down the stack
      v3: - Don't generate a link for Not Run tests in html page
      Tested-by: default avatarBen Widawsky <ben@bwidawsk.net>
      Tested-by: default avatarTom Stellard <thomas.stellard@amd.com>
      Signed-off-by: Dylan Baker's avatarDylan Baker <baker.dylan.c@gmail.com>
    • Daniel Vetter's avatar
      framework: Add support for a test timeout · ab2eb660
      Daniel Vetter authored
      i-g-t tests can take a long time, and for a bunch of reasons (bad
      tuning on disparate set of platforms, stuck in the kernel which often
      can be recovered by interrupting with a signal, ...) that sometimes
      extends to a good approximation of forever.
      Hence this adds a per-test timeout value and a bit of infrastructure
      to adjust the results. Test results adjusting is done after calling
      interpretResult so that we don't have to replicate this all over the
      place. This might need to be adjusted for the piglit-native subtest
      stuff, but otoh igt is a bit special with it's crazy long-running
      tests. So I think we can fix this once it's actually needed.
      The default timeout is None, so this is purely opt-in.
      Note on the implementation: SIG_ALARM doesn't exists on Windows and
      stackoverflow overwhelmingly recommended to go with this thread-based
      approach here. But only tested on my Linux box here.
      I've also timed quick.tests run a bit and the overhead due to the
      additional thread we launch seems to be in the wash. So I didn't opt
      to make the thread launching optional if there's no timeout limit.
      v2: Also add all the boilerplate needed to handle the new test status
      in summaries. For the color I've opted for two shades of light blue,
      they nicely stick out from the current selection.
      v3: Fix GLSLParserTest and ShaderTest. They both derive from
      PlainExecTest but only call the __init__ function of the Test
      baseclass. I haven't really figured why this is and also not really
      what command I should pass to PlainExecTest.__init__, so just
      replicated the init code for now.
      v4: Initialize timeout earlier for tests where we use the ENOENT
      handling to skip them.
      Fix up the out, err passing from the thread. Apparently my idea of how
      python closures work was completely misguided - like with a function
      call a closure captures a copy of of a reference pointing at the
      original object. So assinging anything to them won't have any effect
      outside of the lambda. Hence we need to jump through hoops and pass an
      array around. Nicer fix would be to create a class or something, but
      that seems like overkill.
      v5: Fixup the fraction setting in status.py
      Reviewed-by: Dylan Baker's avatarDylan Baker <baker.dylan.c@gmail.com>
      Cc: Dylan Baker <baker.dylan.c@gmail.com>
      Signed-off-by: Daniel Vetter's avatarDaniel Vetter <daniel.vetter@ffwll.ch>
  13. 05 Nov, 2013 1 commit
  14. 24 Oct, 2013 1 commit
  15. 23 Sep, 2013 1 commit
    • Marek Olšák's avatar
      Optionally capture dmesg changes for each test and report them in a summary · e85194b1
      Marek Olšák authored
      The Radeon driver writes GPU page faults to dmesg and we need to know which
      tests caused them.
      If there is any change in dmesg during a test run, the test result is changed
      as follows:
      * pass -> dmesg-warn
      * warn -> dmesg-warn
      * fail -> dmesg-fail
      Dmesg is captured before and after each test and the difference between the two
      is stored in the test summary.
      The piglit-run.py parameter which enables this behavior is --dmesg. It's also
      recommended to use -c0.
      v2: - address some of Dylan's remarks, mainly bug fixes
          - fix get_dmesg_diff
      Reviewed-by: Dylan Baker's avatarDylan Baken <baker.dylan.c@gmail.com>
  16. 15 Aug, 2013 1 commit
  17. 02 Jul, 2013 2 commits
  18. 31 May, 2013 6 commits
  19. 08 May, 2013 8 commits
  20. 24 Apr, 2012 1 commit
  21. 11 Aug, 2011 1 commit
    • Paul Berry's avatar
      Changed piglit-summary-html to explicitly encode its output as UTF-8. · da3126ce
      Paul Berry authored
      Previously piglit-summary-html didn't explicitly encode its output at
      all--it relied on the default behavior of the Python file.write()
      method (which, in Python 2.x, is to write 8-bit strings verbatim and
      to encode unicode strings using ASCII).  This meant that if a
      non-ASCII character appeared in any of the test data,
      piglit-summary-html would fail with an exception.
      This patch changes piglit-summary-html to explicitly encode its output
      using UTF-8.  It also updates all of the HTML templates to specify
      UTF-8 as the encoding, so that when results are viewed in a web
      browser, non-ASCII characters will display correctly.
  22. 05 Apr, 2011 1 commit
  23. 27 Aug, 2008 1 commit
  24. 30 Jun, 2008 1 commit
  25. 25 Mar, 2007 1 commit