1. 11 Apr, 2018 1 commit
  2. 22 Dec, 2017 1 commit
  3. 13 Jun, 2017 1 commit
  4. 07 Oct, 2016 1 commit
  5. 16 Aug, 2016 2 commits
  6. 10 Aug, 2016 1 commit
  7. 27 May, 2016 1 commit
    • Olivier Berthier's avatar
      framework: Add aborting option when a monitored error is detected · 21031482
      Olivier Berthier authored
      This adds a policy which advises when the user should reboot the system
      to avoid noisy test results due to system becoming unstable, for
      instance, and therefore continues testing successfully.
      
      To do this, a new module is proposed. A class Monitoring is used for
      managing the monitoring rules. Two types of rules, MonitoringFile and
      MonitoringLinuxDmesg, derived from the abstract class MonitoringBase,
      have been implemented. The first allow to track a pattern on standard
      files or locked files. The second, derived from dmesg.LinuxDmesg, will
      track a pattern on the dmesg.
      
      The monitoring rules must be defined in piglit.conf at the section
      monitored-errors. If one of the regex is found, Piglit will raise a
      PiglitAbort exception, stop the test execution -terminating test thread
      pool- and exit with code 3.
      
      Then test execution resume, after rebooting the system or not, is done
      like usually with command line parameter "resume".
      
      To call it, use command line parameter: --abort-on-monitored-error
      
      This option implies --no-concurrent
      
      This include also a set of unit tests for this module.
      Reviewed-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      21031482
  8. 23 May, 2016 1 commit
  9. 15 Mar, 2016 1 commit
    • Jason Ekstrand's avatar
      framework: Add a test runner for the Vulkan CTS · a97a4d6c
      Jason Ekstrand authored
      This is mostly a copy-and-paste of the original GLES3 dEQP runner with a
      few added tidbits that makes running the Vulkan CTS easier.  The two
      force-skip cases were very useful in the early days when lots of tests were
      still in development and may no longer be needed.  However, I have a
      feeling that we will yet want them.
      
      v2 (Dylan):
       - Fix PEP8 issues
       - add and update docstrings
       - update extra_args for global deqp changes
       - update piglit.conf.example
       - remove unused functions
       - use __future__
       - use "bin" instead of "exe" (gles31 is weird that it uses "exe" not
         "bin", all other deqp modules use "bin")
      Reviewed-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      a97a4d6c
  10. 11 Mar, 2016 1 commit
  11. 29 Sep, 2015 1 commit
  12. 09 Jul, 2015 4 commits
    • Dylan Baker's avatar
      framework: add support for lzma via shell calls · 5fdb1639
      Dylan Baker authored
      This requires that there is an xz binary installed, and accessible,
      obviously.
      
      v4: - add this patch
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      5fdb1639
    • Dylan Baker's avatar
      framework: add support for bz2 compression. · 2590bb82
      Dylan Baker authored
      This adds bzip2 compression support to piglit. This produces a file that
      is ~5% the size of the original uncompressed json file.
      
      v4: - add more tests
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      2590bb82
    • Dylan Baker's avatar
      framework: add support for gzip compressionn · 54bf9304
      Dylan Baker authored
      This adds support to compress results with gzip compression. This
      reduces the size of json results significantly (from 21M to 1.6M when
      running the quick profile (which is about 7% of the uncompressed size).
      
      v4: - add additional tests
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      54bf9304
    • Dylan Baker's avatar
      framework: Add ability to set a compression method on file backends · 7be0c99c
      Dylan Baker authored
      This creates a framework for compressing results as part of the
      FileBackend class. This allows for the simple addition of compression
      formats (gz is easily possible in python2, xz in python3), but does not
      actually implement any compression. zip and bz2 compression are also
      possible, although they'll require a little more code.
      
      This patch implements a framework, and tests for that framework.
      
      The goal is that other compression methods can easily be added to piglit
      simply by setting a few values in framework/backends/compression, and
      then everything will just work.
      
      This will allow junit results that are compressed to be read, but
      doesn't add support to junit for compressing results, since junit is
      mainly intended for consumption by jenkins, which doesn't understand
      compression.
      
      v2: - replace tests decorator set_compression with use of
            utils.set_piglit_conf. This is better because it uses a generic
            tools, and because set_compression is somewhat misleading of a
            name.
      v3: - backport changes from my attempt to port this to python3. There
            are significant differences between python2 and python3 compression
            handling because of the bytes/str/unicode differences between the
            two versions. There are also code cleanups from that work here
          - Make a better effort to protect the unit tests from the
            environment around them. The goal is that the tests control the
            compression options 100%, with no chance of piglit.conf or
            environment variables affecting the tests.
          - reduce code duplication in unit tests
          - add additional unit tests as made possible by refactoring changes
      v4: - add additional unit tests
      Signed-off-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      7be0c99c
  13. 01 Jun, 2015 2 commits
  14. 26 Jan, 2015 1 commit
    • Wang Shuo's avatar
      deqp: (V5)Add option to run subset of external dEQP-GLES3 · 656b41d4
      Wang Shuo authored
      Google have already added a subset of dEQP into Android CTS test,
      and we believe this part of dEQP have higher priority than the rest
      of dEQP test cases.
      The case list is stored at some xml files. Such as:
      com.drawelements.deqp.gles3.xml.
      It's git repo lives in the Android tree at
      [https://android.googlesource.com/platform/external/ \
      deqp/+/master/android/cts/com.drawelements.deqp.gles3.xml]
      
      This patch is based on Chad's patch(Add test profile for external
      dEQP-GLES3 tests), and add an option to run the subset of dEQP-GLES3.
      The only differnce for the running method is: you need to set the
      environment variable PIGLIT_DEQP_MUSTPASS or the piglit.conf
      option deqp-gles3.mustpasslist,then it will run the subset of dEQP
      follow the test case list. If not set, it will still run the whole
      dEQP.
      
      Tested on Intel HSW. There are 45866 test cases for the whole dEQP
      and 37354 test cases for the subset of dEQP using Android CTS test
      case list.
      
      V2:
      1.Update the link of get test case list file.
      2.Using xml.etree.cElementTree module instead of re module.
      3.Fix some Piglit style issue
      
      V3:
      1.Update the output issue.
      2.Fix some Piglit style issue.
      
      V4:
      1.Fix some Piglit style issue.
      
      V5:
      1.Fix some Piglit style issue.
      
      v6 (Dylan):
      - Remove trailing whitespace
      Signed-off-by: default avatarWang Shuo <shuo.wang@intel.com>
      Reviewed-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      656b41d4
  15. 22 Dec, 2014 1 commit
    • Chad Versace's avatar
      deqp: Add test profile for external dEQP-GLES3 tests (v4) · 2f122df3
      Chad Versace authored
      Google opensourced the drawElements Quality Product Testsuite (dEQP) as
      part of the Android Lollipop release. It's git repo lives in the Android
      tree at [https://android.googlesource.com/platform/external/deqp/].
      
      This patch adds a new module, deqp_gles3.py, that runs the dEQP-GLES3
      tests.  It queries the 'deqp-gles3' executable at runtime for the list
      of testcases.  The module attempts queries the 'deqp-gles3' executable
      only if the environment variable PIGLIT_DEQP_GLES3_EXE or the
      piglit.conf option deqp-gles3.exe is set.
      
      Why do we need to use Pigit as the testrunner for dEQP? (After all, dEQP has
      its own testrunner). Because dEQP runs all tests in a single process. If test
      17530 of 55409 crashes, then the dEQP testrunner crashes and the remaining
      tests never run. Piglit doesn't suffer from that problem, because it runs each
      test as a separate process.
      
      Tested on Intel Ivybridge by running the command
          piglit run -t dEQP-GLES3/info/vendor \
            tests/deqp_gles3.py tests/quick.py results
      with and without PIGLIT_DEQP_GLES3_EXE set and with and without
      piglit.conf:deqp-gles.exe set. Also tested with various combinations of
      valid and invalid values for PIGLIT_DEQP_GLES_EXTRA_ARGS and
      piglit.conf:deqp-gles.extra_args.
      
      Reviewed-by: Dylan Baker <dylanx.c.baker@intel.com> (v2)
      Reviewed-by: Michael W Mason <michael.w.mason@intel.com> (v4)
      Signed-off-by: default avatarChad Versace <chad.versace@linux.intel.com>
      2f122df3
  16. 25 Sep, 2014 1 commit
    • Mark Janes's avatar
      Filter expected failures from JUnit output · 1921e13d
      Mark Janes authored
      Piglit test results are difficult to analyze with simpler JUnit
      visualizers like those provided by Jenkins.  There are hundreds of
      failures, but the engineer is typically interested only in NEW
      failures.  Jenkins JUnit rules typically expect 100% pass rate, or
      some static threshold of failures.
      
      Specifying test names in the [expected-failures] or [expected-crashes]
      sections of the config file enables JUnit reporting of just the test
      failures which represent new regressions.
      
      Test names are expected in the dotted JUnit format (eg:
      "piglit.spec.ARB_fragment_program.fp-formats") and are case
      insensitive.
      Reviewed-by: default avatarDylan Baker <dylanx.c.baker@intel.com>
      1921e13d
  17. 12 Sep, 2014 1 commit
  18. 02 Sep, 2014 1 commit
  19. 27 Aug, 2014 1 commit
  20. 20 Aug, 2014 1 commit
  21. 10 Jul, 2014 3 commits
  22. 23 Jan, 2014 1 commit
    • Tom Stellard's avatar
      cl: Add support for OpenCV unit tests v3 · 4a13528a
      Tom Stellard authored
      This enables piglit to run and interpret the results from OpenCV's
      gtest based opencv_test_ocl program.
      
      You can enable the OpenCV tests by adding the path to the
      opencv_test_ocl program to your piglit.conf file.
      
      You can also optionally specify the path to OpenCV's work
      directory if you buid OpenCV in a non-standard way.  For example:
      
      [opencv]
      opencv_test_ocl_bindir=/home/user/opencv/build/bin
      opencv_workdir=/home/user/opencv/samples/c/
      
      v2:
        - Python code cleanups
      
      v3:
        - More cleanups
        - Move opencv.py into framework
        - Use ConfigParser
      Reviewed-by: Dylan Baker's avatarDylan Baker <baker.dylan.c@gmail.com>
      4a13528a