1. 23 Jan, 2019 5 commits
  2. 22 Jan, 2019 4 commits
  3. 24 Jul, 2018 1 commit
  4. 17 Apr, 2016 1 commit
  5. 08 Feb, 2016 2 commits
  6. 12 Jan, 2016 1 commit
  7. 25 Nov, 2015 1 commit
    • Feceoru, Gabriel's avatar
      framework: Add support for feature readiness. · f16d011d
      Feceoru, Gabriel authored
      This adds a new "summary feature" command to piglit which creates a HTML table
      with the feature x DUT status (useful for multiple features, multiple DUTs).
      Another use case is the feature status for subsequent test results (depending
      on the meaning of that test result - DUT or build)
      A feature readiness is defined by the piglit regexp which selects the tests
      relevant for that feature and the acceptance percentage threshold (pass rate).
      
      It requires an input json file containing the list of features, in the
      following format (this is just an example):
      {
          "glsl" : {
              "include_tests" : "glsl",
              "exclude_tests" : "",
              "target_rate" : 90
          },
          "arb" : {
              "include_tests" :  "arb_gpu",
              "exclude_tests" : "",
              "target_rate" : 10
          }
      
      }
      
      v3:
      Changed json rate to int instead of string
      Applied other review comments
      
      v2:
      Apply review comments (Dylan, Thomas)
      Fixed 2nd round of review comments
      Signed-off-by: default avatarGabriel Feceoru <gabriel.feceoru@intel.com>
      Reviewed-by: Dylan Baker's avatarDylan Baker <baker.dylan.c@gmail.com>
      f16d011d
  8. 08 Oct, 2015 1 commit
  9. 30 Sep, 2015 1 commit
  10. 24 Sep, 2015 1 commit
  11. 22 Sep, 2015 2 commits
    • Dylan Baker's avatar
      framework/summary.py: drop HTMLIndex class · 598dcb1b
      Dylan Baker authored
      This reduces the amount of code considerably, and doesn't add too much
      complexity to the template. The result is easier to maintain, and that
      the code used is better broken up.
      
      This version is also slightly faster (as measured with two results),
      moving from ~1.8 seconds to ~1.2
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      598dcb1b
    • Dylan Baker's avatar
      framework: replace TestResult dict with an object · b365367f
      Dylan Baker authored
      This is a very invasive patch, because it replaces one of our core
      data-structures with a completely different kind of object. This new
      object is not a dict-like object (it doesn't use obj[key] = value
      syntax), instead it's a standard object with the standard object
      attributes. So result['time'] becomes result.time.
      
      This approach has a couple of advantages. First, it allows us to use
      properties, which allows us to encapsulate a lot of distributed logic
      from the summary module in the results module, and in a way that is
      easier to test for correctness. The second advantage of that
      encapsulation is that correct behavior is used everywhere, instead of
      just in most places. Finally, it allows us to use the flyweight pattern
      on the results objects, limiting the amount of memory consumed.
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      b365367f
  12. 05 May, 2015 1 commit
    • Dylan Baker's avatar
      framework: add incomplete status. · e62cfdac
      Dylan Baker authored
      This will be used later in this series
      
      This status has a background color of purple in the html summary, and is
      considered the worst status.
      
      My logic for making it the worst status is as follows: basically we
      always want this to show up as a regression, because it's a very unusual
      status (only available when piglit is aborted or when the whole system
      goes down).
      
      v2: - use white text instead of black, it makes things infinitely more
            readable
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      Reviewed-by: Mark Janes's avatarMark Janes <mark.a.janes@intel.com>
      e62cfdac
  13. 03 Sep, 2014 1 commit
  14. 11 Jun, 2014 1 commit
  15. 19 May, 2014 1 commit
  16. 28 Apr, 2014 1 commit
  17. 24 Apr, 2014 1 commit
    • Dylan Baker's avatar
      tests/xts.py: Add a tests file for the X Test suite. · e1057c4a
      Dylan Baker authored
      This adds support for running XTS from piglit.
      
      This patch was original written by Eric Anholt, but I've cleaned it up
      and reworked parts of it for upstreaming, and as a result I've attached
      my name as the author so ``git bisect'' and ``git blame'' will send
      angry mail my way instead of Eric's.
      
      v2: - Don't break dmesg reporting
          - Use str.format consistently
          - populate profile after check for xtest symlink
      v3: - rebase against master
          - fix spelling errors
      Signed-off-by: Dylan Baker's avatarDylan Baker <baker.dylan.c@gmail.com>
      Reviewed-by: Eric Anholt's avatarEric Anholt <eric@anholt.net>
      e1057c4a
  18. 23 Apr, 2014 1 commit
  19. 18 Apr, 2014 2 commits
  20. 23 Jan, 2014 1 commit
  21. 15 Nov, 2013 1 commit
  22. 14 Nov, 2013 2 commits
    • Dylan Baker's avatar
      summary.py: Treat subtests as groups · e0d4bdba
      Dylan Baker authored
      This patch causes tests with subtests to be treated as a group, rather
      than as a test. This means that the status the test itself stores will
      be overwritten by those in the subtest.
      
      There is one oddity about this to be aware of; a test with subtests that
      crashes or fails before any of the subtests run will report a fraction
      of 0/1 with the appropriate color, even though all of the subtests will
      report Not Run.
      
      v2: - Add subtests to the results file as full tests (the internal view
            of the json), without this they will not appear in changes, fixes,
            etc
          - Render the background color of Not Run tests correctly in HTML
          - Apply subtest fractions down the stack
      v3: - Don't generate a link for Not Run tests in html page
      Tested-by: default avatarBen Widawsky <ben@bwidawsk.net>
      Tested-by: default avatarTom Stellard <thomas.stellard@amd.com>
      Signed-off-by: Dylan Baker's avatarDylan Baker <baker.dylan.c@gmail.com>
      e0d4bdba
    • Daniel Vetter's avatar
      framework: Add support for a test timeout · ab2eb660
      Daniel Vetter authored
      i-g-t tests can take a long time, and for a bunch of reasons (bad
      tuning on disparate set of platforms, stuck in the kernel which often
      can be recovered by interrupting with a signal, ...) that sometimes
      extends to a good approximation of forever.
      
      Hence this adds a per-test timeout value and a bit of infrastructure
      to adjust the results. Test results adjusting is done after calling
      interpretResult so that we don't have to replicate this all over the
      place. This might need to be adjusted for the piglit-native subtest
      stuff, but otoh igt is a bit special with it's crazy long-running
      tests. So I think we can fix this once it's actually needed.
      
      The default timeout is None, so this is purely opt-in.
      
      Note on the implementation: SIG_ALARM doesn't exists on Windows and
      stackoverflow overwhelmingly recommended to go with this thread-based
      approach here. But only tested on my Linux box here.
      
      I've also timed quick.tests run a bit and the overhead due to the
      additional thread we launch seems to be in the wash. So I didn't opt
      to make the thread launching optional if there's no timeout limit.
      
      v2: Also add all the boilerplate needed to handle the new test status
      in summaries. For the color I've opted for two shades of light blue,
      they nicely stick out from the current selection.
      
      v3: Fix GLSLParserTest and ShaderTest. They both derive from
      PlainExecTest but only call the __init__ function of the Test
      baseclass. I haven't really figured why this is and also not really
      what command I should pass to PlainExecTest.__init__, so just
      replicated the init code for now.
      
      v4: Initialize timeout earlier for tests where we use the ENOENT
      handling to skip them.
      
      Fix up the out, err passing from the thread. Apparently my idea of how
      python closures work was completely misguided - like with a function
      call a closure captures a copy of of a reference pointing at the
      original object. So assinging anything to them won't have any effect
      outside of the lambda. Hence we need to jump through hoops and pass an
      array around. Nicer fix would be to create a class or something, but
      that seems like overkill.
      
      v5: Fixup the fraction setting in status.py
      Reviewed-by: Dylan Baker's avatarDylan Baker <baker.dylan.c@gmail.com>
      Cc: Dylan Baker <baker.dylan.c@gmail.com>
      Signed-off-by: Daniel Vetter's avatarDaniel Vetter <daniel.vetter@ffwll.ch>
      ab2eb660
  23. 05 Nov, 2013 1 commit
  24. 24 Oct, 2013 1 commit
  25. 23 Sep, 2013 1 commit
    • Marek Olšák's avatar
      Optionally capture dmesg changes for each test and report them in a summary · e85194b1
      Marek Olšák authored
      The Radeon driver writes GPU page faults to dmesg and we need to know which
      tests caused them.
      
      If there is any change in dmesg during a test run, the test result is changed
      as follows:
      * pass -> dmesg-warn
      * warn -> dmesg-warn
      * fail -> dmesg-fail
      Dmesg is captured before and after each test and the difference between the two
      is stored in the test summary.
      
      The piglit-run.py parameter which enables this behavior is --dmesg. It's also
      recommended to use -c0.
      
      v2: - address some of Dylan's remarks, mainly bug fixes
          - fix get_dmesg_diff
      Reviewed-by: Dylan Baker's avatarDylan Baken <baker.dylan.c@gmail.com>
      e85194b1
  26. 15 Aug, 2013 1 commit
  27. 02 Jul, 2013 2 commits
  28. 31 May, 2013 1 commit