Skip to content

nmci/misc.py: saner search for feature file containing test tag; README.md update

David Jaša requested to merge dj/nmci-load-tags-respect-comments-anywhere into main

Recently @general_nmcli_offline_connection_add_modify test started failing as duplicate test for no apparent reason, it was defined twice but with correct @ver* tags. Eventually @vbenes found the reason , it was my change introducing this line to doc_procedures.py:

    # the same feature as @general_nmcli_offline_connection_add_modify tests

so he fixed it in commit dbb7bc18 by removing @ from there.

The reason why the test was evaluated by duplicate was actual check in misc.py's function test_find_feature_file() which literally runs:

grep '@\<{test_name}\>' -l -- FEATURE_FILES

so this was only one of possible incorrect failure modes, another being two scenarios in different files of which one is subset of the other (say @pass in general.feature and hypothetical @pass_some_value somewhere else). Therefore I'm proposing this change which actually matches only valid test tag by looking for sequence 'newline-spaces-testtag-newline-spaces-Scenario:'. It's somewhat slower on my test machine (for 1000 timeit cycles which should roughly match number of function runs during whole CI run):

original:               native:
1.957155727001009       3.0595776560003287
1.8704291880003439      3.052115333000984
2.4855815279988747      3.081021817999499
1.5514806110004429      3.0857457980000618
2.257825339998817       3.0868119899987505
1.6917649849983718      3.0788737590009987
1.5966649700003472      3.113616422000632
1.8755290230001265      3.0739072389987996
2.291331889999128       3.06967529099893
1.7524898240008042      3.16748733500026

but the correct behaviour should be a priorityhere IMO. (Numbers are from benchmark timeit_get_ff.py that just replaces arguments to the function with fixed variables).

When touching README.md, I somehow went graphomaniac so there are some non-negligible additions to it as well.

Edited by David Jaša

Merge request reports