Skip to content
Snippets Groups Projects
Commit fe06ddff authored by Janusz Krzysztofik's avatar Janusz Krzysztofik
Browse files

lib/ktap: Reimplement KTAP parser


Current implementation of KTAP parser suffers from several issues:
- works only with built-in kunit, can't parse KTAP output if modular,
- in most cases, kernel messages that are not part of KTAP output but
  happen to appear in between break the parser,
- results from parametrized test cases, not preceded with a "1..N" test
  plan, break the parser,
- skips are not supported, reported as success,
- IGT results from all 3 kunit test nesting levels, i.e., from
  parametrized subtests (should those were fixed to work correctly), test
  cases and test suites, are reported individually as if all those items
  were executed sequentially, all at the same level of nesting, which can
  be confusing to igt_runner,
- subtest names mostly consist of kunit suite name and kunit test case
  name but not always, sometimes the first component is omited,
- the parser is not only parsing the data, but also handles data input
  from a /dev/kmsg like source, which is integrated into it, making it
  hard if not impossible to feed KTAP data from different sources,
  including mock sources,
- since the parser has been designed for running it in a separate thread,
  it's not possible to use igt_skip() nor igt_fail() and friends
  immediately when a result is available, only pass it to the main thread
  over a buffer.  As a consequence, it is virtually impossible to
  synchronize IGT output with KTAP output.

Fixing the existing parser occurred more complicated than re-implementing
it from scratch.  This patch provides a new implementation.

Only results from kunit test cases are reported as results of IGT dynamic
sub-subtests.  Results from individual parametrized subtests have been
considered problematic since many of them provide multi-word descriptions
in place of single-word subtest names.  If a parametrized test case fails
then full KTAP output from its subtests, potentially mixed with
accompanying kernel messages, is available in dmesg for analysis so users
can still find out which individual subtests succeeded and which failed.

Results from test suites level are also omitted in faith that IGT can
handle aggregation of results from individual kunit test cases reported as
IGT dynamic sub-subtests and report those aggregated results correctly as
results from an IGT dynamic subtest.  That 1:1 mapping of kunit test
suites to IGT dynamic subtests now works perfectly for modules that
provide only one test suite, which is the case for all kunit test modules
now existing under drivers/gpu/drm, and the most common case among all
kunit test modules in the whole kernel tree.

New igt_ktap functions can be called directly from igt_kunit subtest body,
but for this patch, the old igt_ktap_parser() function that runs in a
separate thread has been preserved, only modified to use the new
implementation and translate results from those new functions to legacy
format.  Unlike the former implementation, translation of kunit names to
IGT names is handled outside the parser itself, though for now it is still
performed inside the legacy igt_ktap_parser() function.

For better readability of the patch, no longer used functions have been
left untouched, only tagged with __maybe_unused to shut up compiler
warnings / errors.  Kunit library functions will be modified to use the
new igt_ktap interface, and those old ktap functions removed by follow-
up patches.

A test with example subtests that feed igt_ktap_parse() function with some
example data and verifies correctness of their parsing is also provided.

v2: Fix incorrect and missing includes in the test source file,
  - add license and copyright clauses to the test source file.

Signed-off-by: default avatarJanusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Acked-by: default avatarMauro Carvalho Chehab <mchehab@kernel.org>
parent 1674d77a
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment