Peter Senna Tschudin authored 2 weeks ago and Kamil Konieczny committed 2 weeks ago

This patch modifies igt_runner to support runner_kmemleak() calls. By
default, kmemleak scanning is disabled, so new command-line options are
introduced to enable it:
 * -k, -k<option>, --kmemleak, --kmemleak=<option>

The available options are:
 * once: Do single kmemleak scan after last test in the test list
 * each: Perform a kmemleak scan after each test completes

By default, kmemleak scanning is disabled. If any kmemleaks are
detected, they will be saved in the igt_runner results directory under
kmemleak.txt.

Additionally, this patch updates serialize_settings() and
read_settings_from_file() to persist igt_runner settings across runs.
This allows settings to be saved when running igt_runner --dry-run and
later restored when executing igt_resume.

The unit tests for igt_runner have been extended to verify:
 * Kmemleak scans are disabled by default
 * Kmemleak scans can be enabled via command-line arguments
 * The kmemleak setting is correctly saved to and restored from disk

To test the new -k command-line option, this patch appends "--overwrite"
to *argv[] in runner_test.c to expand the argument array. This approach
avoids a major refactor of how *argv[] is defined across the file while
keeping the changes isolated to unit testing. Since this only affects
tests, there is no downstream impact.

Cc: vitaly.prosyak@amd.com
Cc: christian.koenig@amd.com
Cc: alexander.deucher@amd.com
Cc: jesse.zhang@amd.com
Cc: harry.wentland@amd.com
Cc: zbigniew.kempczynski@intel.com
Cc: kamil.konieczny@linux.intel.com
Cc: ryszard.knop@intel.com
Cc: lucas.demarchi@intel.com
Cc: katarzyna.piecielska@intel.com
Reviewed-by: Zbigniew Kempczyński <zbigniew.kempczynski@intel.com>
Reviewed-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Reviewed-by: Vitaly Prosyak <vitaly.prosyak@amd.com>
Signed-off-by: Peter Senna Tschudin <peter.senna@linux.intel.com>

Serialize the command line to metadata.txt. The expected format in the
metadata.txt is like below:

	cmdline.argc : 6
	cmdline.argv[0] : ./build/runner/igt_runner
	cmdline.argv[1] : -o
	cmdline.argv[2] : --test-list
	cmdline.argv[3] : /tmp/testlist.txt
	cmdline.argv[4] : build/tests/
	cmdline.argv[5] : /tmp/results

Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Tested-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Reviewed-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Link: https://lore.kernel.org/r/20250207231039.2883195-11-lucas.demarchi@intel.com


Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

Prepare parser/serialize to handle arrays.

Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Tested-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Reviewed-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Link: https://lore.kernel.org/r/20250207231039.2883195-10-lucas.demarchi@intel.com


Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

Currently there's no support for newlines on arguments passed to runner.
However it's also a silent failure:

	# igt_runner --test-list '/tmp/test
	list2.txt' build/tests/ /tmp/results

	# head /tmp/results/metadata.txt
	disk_usage_limit : 0
	test_list : /tmp/test
	list2.txt
	name : results
	...

	# ./build/runner/igt_resume /tmp/results
	[9840425.334900] All tests already executed.
	resume failed at generating results
	Done.

Embedding a newline like this is very dubious for test-list, but it's
used for e.g. hooks. In future we will add the command line to the
metadata and possibly migrate the hooks, so add support for
escaping/unescaping the string on save/restore.

The method chosen is slightly different than the one used for hooks:
instead of adding a escape char and keeping the char escaped, this just
prefers using an hex representation of the char with a \x<HEX>h
sequence. This makes it easier when unescaping since the reader can
continue reading one line per iteration. In future this can also be
adopted by the hooks or even migrating the hooks to use metadata.txt.

Another fix is that now we just skip null values on the serialization
side.  Previously it would serialize "(null)" and then load that string
instead of NULL. Add code_coverage_script to the runner_test to cover
that, which would previously fail.

Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Tested-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Reviewed-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Link: https://lore.kernel.org/r/20250207231039.2883195-9-lucas.demarchi@intel.com


Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

No need to strdup() again since the fscanf() function is already
allocating the variable. Just set the pointer to NULL so we "leak" our
variable to be saved in the settings.

Tested-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Reviewed-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://lore.kernel.org/r/20250207231039.2883195-8-lucas.demarchi@intel.com


Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

Use similarly named macros on both sides of serialize/parse.

Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Tested-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Reviewed-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Link: https://lore.kernel.org/r/20250207231039.2883195-7-lucas.demarchi@intel.com


Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

Simplify assigning the variables by using functions called by wrapper
macros. This avoids calling atoi() on every iteration and will help
future refactors on functions parsing the values.

The pointer to the value is passed to the parse function since it will
be useful later when parsing a string and leaking it to the settings
struct rather than duplicating.

Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Tested-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Reviewed-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Link: https://lore.kernel.org/r/20250207231039.2883195-6-lucas.demarchi@intel.com


Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

Deduplicate cleanup so it's also easy to parse the line in a different
way.

Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Tested-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Reviewed-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Link: https://lore.kernel.org/r/20250207231039.2883195-5-lucas.demarchi@intel.com


Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

Make sure to free it when clearing settings.

Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Tested-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Reviewed-by: Peter Senna Tschudin <peter.senna@linux.intel.com>
Link: https://lore.kernel.org/r/20250207231039.2883195-3-lucas.demarchi@intel.com


Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>

Peter Senna Tschudin authored 3 months ago and Kamil Konieczny committed 3 months ago

Modifies igt_runner to include calls to igt_facts() before the
execution of each test and after the final test concludes. Facts are
disabled by default, so add command line options to igt_runner to
enable facts:  -f, --facts

Updates serialize_settings() and read_settings_from_file() to save
and restore igt_runner settings to and from disk. This is used when
calling igt_runner with '--dry-run' and then by calling igt_resume
instead of igt_runner.

Updates unit testing for igt_runner to test that:
 - Facts are disabled by default
 - Facts can be enabled by command line arguments
 - The choice about facts being enabled or not is saved to disk
   and restored from disk

CC: Helen Koike <helen.koike@collabora.com>
CC: Jani Nikula <jani.nikula@linux.intel.com>
CC: Jani Saarinen <jani.saarinen@intel.com>
CC: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
CC: Juha-Pekka Heikkila <juha-pekka.heikkila@intel.com>
CC: Kamil Konieczny <kamil.konieczny@linux.intel.com>
CC: Lucas De Marchi <lucas.demarchi@intel.com>
CC: Maíra Canal <mcanal@igalia.com>
CC: Melissa Wen <mwen@igalia.com>
CC: Petri Latvala <adrinael@adrinael.net>
CC: Rob Clark <robdclark@chromium.org>
CC: Ryszard Knop <ryszard.knop@intel.com>
CC: Swati Sharma <swati2.sharma@intel.com>
CC: Zbigniew Kempczyński <zbigniew.kempczynski@intel.com>
CC: dominik.karol.piatkowski@intel.com
CC: himal.prasad.ghimiray@intel.com
CC: katarzyna.piecielska@intel.com
CC: luciano.coelho@intel.com
CC: nirmoy.das@intel.com
CC: stuart.summers@intel.com
Reviewed-by: Zbigniew Kempczyński <zbigniew.kempczynski@intel.com>
Reviewed-by: Ryszard Knop <ryszard.knop@intel.com>
Reviewed-by: Dominik Karol Piątkowski <dominik.karol.piatkowski@intel.com>
Signed-off-by: Peter Senna Tschudin <peter.senna@linux.intel.com>

In testing drm-tip it is helpful to have reported all tests
results, especially when one have many dynamic ones we want
to get a parent result.

v2: fix for runner_tests (Jari)

Cc: Ewelina Musial <ewelina.musial@intel.com>
Cc: Helen Koike <helen.koike@collabora.com>
Cc: Jari Tahvanainen <jari.tahvanainen@intel.com>
Cc: Petri Latvala <adrinael@adrinael.net>
Cc: Rob Clark <robdclark@gmail.com>
Signed-off-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
Reviewed-by: Peter Senna Tschudin <peter.senna@intel.com>
Tested-by: Jari Tahvanainen <jari.tahvanainen@intel.com>

Gustavo Sousa authored 7 months ago and Matt Roper committed 7 months ago

Test binaries now allow passing multiple --hook options, it just makes
sense that igt_runner follows suit, so let's do it.

Note that this requires having another file in the results directory for
storing the hook strings, as metadata.txt does not support multivalued
items.

Since we are using a different file to store hook strings, take this
opportunity to also allow multiline hook strings (which was not possible
with metadata.txt).

Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://lore.kernel.org/r/20240814204822.95283-7-gustavo.sousa@intel.com


Signed-off-by: Matt Roper <matthew.d.roper@intel.com>

Gustavo Sousa authored 7 months ago and Matt Roper committed 7 months ago

Extend the current hook functionality to allow using multiple hook
descriptors. That allows running a test binary like the following:

    my-test --hook pre-subtest:do-something \
            --hook post-subtest:do-somthing-else

Which is more convenient to the user than having to implement a script
that checks the value of IGT_HOOK_EVENT environment variable.

Note that we still need to add the same support for igt_runner, which is
left for a followup change.

Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://lore.kernel.org/r/20240814204822.95283-6-gustavo.sousa@intel.com


Signed-off-by: Matt Roper <matthew.d.roper@intel.com>

Gustavo Sousa authored 7 months ago and Matt Roper committed 7 months ago

Now that we have support for setting a hook script for test cases, let's
also add the option --hook to igt_runner, which forwards it to test
executables.

Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://lore.kernel.org/r/20240814204822.95283-4-gustavo.sousa@intel.com


Signed-off-by: Matt Roper <matthew.d.roper@intel.com>

There are some KMS tests that are case-sensitive. Eventually,
those could be blocklisted with a different case, causing troubles
for CI runs.

As it makes no sense to have the same test name with different
cases, handle regular expressions in a case-insensitive way.

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

When --dry-run option is used, there's no need to require root,
as no tests will actually be executed.

Reviewed-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Per Mauro's review, add some --environment tests:
- Check if -e/--environment works,
- Check both KEY=VALUE and KEY,
- Check if resumes load the variables correctly.

Settings serialization now creates a separate file, environment.txt,
which contains KEY=VALUE pairs of environment variables to use for test
process spawning. Only vars created with --environment are serialized.
Also add misc warning fixes.

v2:
- Simplify env var reading from files (Mauro)
- Don't align settings loading block (Mauro)
- Make filenames static const (Mauro)

Allows specifying additional environment variables to be set for the
launched test processes. This commit introduces the argument parsing.

v2:
- Moved warning fixes from later commits here
- igt_list_empty_or_null -> igt_list_empty (Mauro)
- Optimized the parser, more error handling (Mauro)

For easier error logging, make the usage() function variadic and pass
its arguments to vfprintf. Additionally, reorder its arguments so that
it visually matches all the printf functions.

v2:
- usage(msg, ...) -> usage(stderr) (Maira)
- __attribute__ ((format (...))) (Petri)

That function does not actually free the settings pointer - rename it
for clarity.

Mauro Carvalho Chehab authored 2 years ago and Petri Latvala committed 2 years ago

In order to prepare to execute the code coverage scripts from the
PATH, change the logic at settings in order for it to seek for
the script in the PATH, if it doesn't contain any directories
on its filename.

Note: file search routines were moved (unchanged) on this path,
in order to avoid forward prototype declarations.

Reviewed-by: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Allow finer control of reporting dynamic subtests instead of
unconditionally assuming that the main subtest result and logs are
uninteresting if the subtest has dynamic subtests.

The default is still to remove subtest results when the subtest has
dynamic subtests. Other options are:

keep-subtests: Remove the dynamic subtests instead, for cases when a
stable test count is more important.

keep-all: Remove nothing.

keep-requested: Remove the results that were not directly requested to
be executed. This option is useful in cases where the test selection
is a hand-picked mix of subtests and particular dynamic subtests.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arek@hiler.eu>
Reviewed-by: Swati Sharma <swati2.sharma@intel.com>

Mauro Carvalho Chehab authored 3 years ago and Petri Latvala committed 3 years ago

The gcc compiler has a feature that enables checking the code coverage
in runtime[1].

[1] See https://www.kernel.org/doc/html/latest/dev-tools/gcov.html



The Linux Kernel comes with an option to enable such feature:

	./scripts/config -e DEBUG_FS -e GCOV_KERNEL

The driver's Makefile also needs change to enable it. For instance, in
order to enable GCOV for all DRM drivers, one would need to run:

	for i in $(find drivers/gpu/drm/ -name Makefile); do
                sed '1 a GCOV_PROFILE := y' -i $i
        done

This patch adds support for it by:

a) Implementing a logic to cleanup the code coverage counters via sysfs;
b) Calling a script responsible for collecging code coverage data.

The implementation works with two modes:

1) It zeroes the counters, run all IGT tests and collects the code
   coverage results at the end.

   This implies that no tests would crash the driver, as otherwise the
   results won't be collected;

   This is faster, as collecting code coverage data can take several
   seconds.

2) For each test, it will clean the code coverage counters, run the and
   collect the results.

   This is more reliable, as a Kernel crash/OOPS won't affect the
   results of the previously ran tests.

Reviewed-by: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Mauro Carvalho Chehab authored 3 years ago and Petri Latvala committed 3 years ago

Without root permissions, most IGT tests won't actually run, but they
would be displayed at the runner's output as if everything went fine.

In order to avoid that, check if one attempts to run IGT without root
permission. Such check can be disbled with a new command line option:

	--allow-non-root

As runner_tests runs as non-root, most unit tests need to pass
--allow-non-root in order for them to not return an error.

Reviewed-by: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

Disk usage limit is a limit of disk space taken, per (dynamic)
subtest. If the test's output, kernel log included, exceeds this
limit, the test is killed, similarly to killing the test when the
kernel gets tainted.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>

Noticed while running some tests that adding any kind of spaces into the
name of a test run would stop igt_resume from working for said test run.
Turns out that when we parse test metadata, we accidentally use the
'%ms' specifier with fscanf() which finishes parsing strings when any
kind of whitespace is encountered.

So, fix this by using the proper %m[^\n] specifier, which dynamically
allocates it's result and doesn't stop reading the string until a
newline is encountered. Additionally, add a test for this.

Signed-off-by: Lyude Paul <lyude@redhat.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>

To help verify correct deployment, add a --version flag to igt_runner
that just prints the IGT-version text, the same tests would print.

Note that only igt_runner gained the --version flag. igt_resume and
igt_results don't do fancy flag handling, they only accept the
directory to operate as their single arg.

v2: Depend on version.h (CI)

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Tomi Sarvela <tomi.p.sarvela@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>

A new config option, --per-test-timeout, sets a time a single test
cannot exceed without getting itself killed. The time resets when
starting a subtest or a dynamic subtest, so an execution with
--per-test-timeout=20 can indeed go over 20 seconds a long as it
launches a dynamic subtest within that time.

As a bonus, verbose log level from runner now also prints dynamic
subtest begin/result.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

If the network goes down while testing, CI tends to interpret that as
the device being down, cutting its power after a while. This causes an
incomplete to an innocent test, increasing noise in the results.

A new flag to --abort-on-monitored-error, "ping", uses liboping to
ping a host configured in .igtrc with one ping after each test
execution and aborts the run if there is no reply in a hardcoded
amount of time.

v2:
 - Use a higher timeout
 - Allow hostname configuration from environment
v3:
 - Use runner_c_args for holding c args for runner
 - Handle runner's meson options in runner/meson.build
 - Instead of one ping with 20 second timeout, ping with 1 second timeout
   for a duration of 20 seconds
v4:
 - Rebase
 - Use now-exported igt_load_igtrc instead of copypaste code
 - Use define for timeout, clearer var name for single attempt timeout

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Cc: Tomi Sarvela <tomi.p.sarvela@intel.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>

If provided an empty blacklist let's fail instead of returning a value
of uninitialized variable.

Cc: Oleg Vasilev <oleg.vasilev@intel.com>
Cc: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>

Oleg Vasilev authored 5 years ago and Petri Latvala committed 5 years ago

Currently, runner already collects all subtest names into job_list.
--list-all allows to output it to stdout.

--blacklist option takes a filename as an argument and adds all regexes
from that file to the exclusion list.

v2:
 - Update exclude/include regex matches for tests without
	subtests to be matched with pigtit-like name (Petri)
 - Replace relative paths with those formatted with testdatadir (Petri)
 - Minor codestyle changes

v3:
 - Print test names in lowercase (Petri)

v4:
 - Replaced custom str_to_lower with generate_piglit_name (Petri)
 - Minor codestyle changes

Cc: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Oleg Vasilev <oleg.vasilev@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>

POSIX ERE, while pretty standard is also very old and severely limited.
In fact, it's so limited that Intel CI's own test blacklist,
tests/intel-ci/blacklist.txt, doesn't even work with it due to the lack
of support for backreferences.

Since we're already using glib in other parts of igt, let's just start
using glib's regular expression matching instead of the POSIX regex API.
This gives us full perl compatible regular expressions and in turn, also
gives us python compatible regular expressions to match piglit.

Reviewed-by: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Lyude Paul <lyude@redhat.com>

This switch allows users to select which dmesg log level is treated as
warning resulting in overriding the test results to
dmesg-fail/dmesg-warn.

Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>

To aid testing function parsing metadata.txt is split into outer helper
that operates on dirfd and inner function that operates on FILE*.

This allows us to test the parsing using fmemopen(), limiting the amount
of necessary boilerplate.

Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>

Deviating a bit from the piglit command line flag, igt_runner takes an
optional comma-separated list as an argument to
--abort-on-monitored-error for the list of conditions to abort
on. Without a list all possible conditions will be checked.

Two conditions implemented:
 - "taint" checks the kernel taint level for TAINT_PAGE, TAINT_DIE and
 TAINT_OOPS
 - "lockdep" checks the kernel lockdep status

Checking is done after every test binary execution, and if an abort
condition is met, the reason is printed to stderr (unless log level is
quiet) and the runner doesn't execute any further tests. Aborting
between subtests (when running in --multiple-mode) is not done.

v2:
 - Remember to fclose
 - Taints are unsigned long (Chris)
 - Use getline instead of fgets (Chris)
v3:
 - Fix brainfart with lockdep
v4:
 - Rebase
 - Refactor the abort condition checking to pass down strings
 - Present the abort result in results.json as a pseudo test result
 - Unit tests for the pseudo result
v5:
 - Refactors (Chris)
 - Don't claim lockdep was triggered if debug_locks is not on
   anymore. Just say it's not active.
 - Dump lockdep_stats when aborting due to lockdep (Chris)
 - Use igt@runner@aborted instead for the pseudo result (Martin)
v6:
 - If aborting after a test, generate results.json. Like was already
   done for aborting at startup.
 - Print the test that would be executed next as well when aborting,
   as requested by Tomi.
v7:
 - Remove the resolved TODO item from commit message

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Tomi Sarvela <tomi.p.sarvela@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>

With --overall-timeout $foo, the runner will stop executing new tests
when $foo seconds have already been used.

A resumed run will start over with no time used, using the same
timeout. This allows for executing a long list of tests piecemeal, in
about $foo length executions.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=106127


Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Tomi Sarvela <tomi.p.sarvela@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>

With the flag, dmesg handling is done exactly as piglit does it: Level
5 (info) and higher dmesg lines, if they match a regexp, cause test
result to change to dmesg-*.

The default is false (use new method).

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Acked-by: Martin Peres <martin.peres@linux.intel.com>

absolute_path() tends to return NULL if more than the last element of
the path is nonexistent. That behavior is confusing the callers, which
use NULL as a convention for something not being set at all.

Let's fix that by sprinkling a little bit of recursion onto
absolute_path() and wrapping POSIX in some sanity.

Cc: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Petri Latvala <petri.latvala@intel.com>