Skip to content

behave unit test: reprint stdout+err if there's no UNFINISHED line in stdout

David Jaša requested to merge dj/behave-reprint-on-different-error into main

missing dependency makes the test currently fail with no clear error. While running behave by hand reveals what's wrong easily, we should not make diagnosing any harder than necessary.

Missing pyte module before:

$ pytest -k behave
================================================= test session starts ==================================================
platform linux -- Python 3.11.1, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/home/djasa/src/NetworkManager-ci, configfile: pyproject.toml
collected 42 items / 41 deselected / 1 selected                                                                        

nmci/test_nmci.py F                                                                                              [100%]

======================================================= FAILURES =======================================================
__________________________________________ test_behave_steps_in_feature_files __________________________________________

capfd = <_pytest.capture.CaptureFixture object at 0x7f2a325bac50>

    def test_behave_steps_in_feature_files(capfd):
        b_cli = ["behave", "-d", "-c", "--no-summary", "--no-snippets", "-f", "steps.usage"]
        try:
            proc = subprocess.run(b_cli)
        except FileNotFoundError:
            pytest.skip("behave is not available for check if all the steps are recognized")
    
        cap = capfd.readouterr()
        assert len(cap.out) > 0
        undefs_absent = print_undefs(cap.out)
        assert undefs_absent, "Following undefined steps were encountered:"
>       assert proc.returncode == 0, "behave ended up with non-zero return code"
E       AssertionError: behave ended up with non-zero return code
E       assert 1 == 0
E        +  where 1 = CompletedProcess(args=['behave', '-d', '-c', '--no-summary', '--no-snippets', '-f', 'steps.usage'], returncode=1).returncode

nmci/test_nmci.py:2031: AssertionError
=============================================== short test summary info ================================================
FAILED nmci/test_nmci.py::test_behave_steps_in_feature_files - AssertionError: behave ended up with non-zero return code
=========================================== 1 failed, 41 deselected in 0.40s ===========================================

and after:

$ git checkout dj/behave-reprint-on-different-error 
Switched to branch 'dj/behave-reprint-on-different-error'
Your branch is up to date with 'origin/dj/behave-reprint-on-different-error'.
$ pytest -k behave
================================================= test session starts ==================================================
platform linux -- Python 3.11.1, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/home/djasa/src/NetworkManager-ci, configfile: pyproject.toml
collected 42 items / 41 deselected / 1 selected                                                                        

nmci/test_nmci.py F                                                                                              [100%]

======================================================= FAILURES =======================================================
__________________________________________ test_behave_steps_in_feature_files __________________________________________

capfd = <_pytest.capture.CaptureFixture object at 0x7f61e5ff4a90>

    def test_behave_steps_in_feature_files(capfd):
        b_cli = ["behave", "-d", "-c", "--no-summary", "--no-snippets", "-f", "steps.usage"]
        try:
            proc = subprocess.run(b_cli)
        except FileNotFoundError:
            pytest.skip("behave is not available for check if all the steps are recognized")
    
        cap = capfd.readouterr()
        assert len(cap.out) > 0
        undefs_absent = print_undefs(cap)
        assert undefs_absent, "Following undefined steps were encountered:"
>       assert proc.returncode == 0, "behave ended up with non-zero return code"
E       AssertionError: behave ended up with non-zero return code
E       assert 1 == 0
E        +  where 1 = CompletedProcess(args=['behave', '-d', '-c', '--no-summary', '--no-snippets', '-f', 'steps.usage'], returncode=1).returncode

nmci/test_nmci.py:2035: AssertionError
------------------------------------------------- Captured stdout call -------------------------------------------------
Exception ModuleNotFoundError: No module named 'pyte'
------------------------------------------------- Captured stderr call -------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/bin/behave", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/behave/__main__.py", line 183, in main
    return run_behave(config)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/behave/__main__.py", line 127, in run_behave
    failed = runner.run()
             ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/behave/runner.py", line 804, in run
    return self.run_with_paths()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/behave/runner.py", line 808, in run_with_paths
    self.load_hooks()
  File "/usr/local/lib/python3.11/site-packages/behave/runner.py", line 784, in load_hooks
    exec_file(hooks_path, self.hooks)
  File "/usr/local/lib/python3.11/site-packages/behave/runner_util.py", line 386, in exec_file
    exec(code, globals_, locals_)
  File "features/environment.py", line 13, in <module>
    from features.steps.nmtui import get_cursored_screen, log_tui_screen
  File "/var/home/djasa/src/NetworkManager-ci/features/steps/nmtui.py", line 2, in <module>
    import pyte
ModuleNotFoundError: No module named 'pyte'
=============================================== short test summary info ================================================
FAILED nmci/test_nmci.py::test_behave_steps_in_feature_files - AssertionError: behave ended up with non-zero return code
=========================================== 1 failed, 41 deselected in 0.37s ===========================================

RunTests:pass

Merge request reports