Commit d4093a3a authored by Thomas Haller's avatar Thomas Haller

clients/tests: add python test script for nmcli tests

Add a test which runs nmcli against our stub NetworkManager
service and compares the output.

The output formats of nmcli are complicated and not easily understood.
For example how --mode tabular|multiline interacts with selecting
output-fields (--fields) and output modes ([default]|--terse|--pretty).
Also, there are things like `nmcli connection show --order $FIELD_SPEC`.

We need unit tests to ensure that we don't change the output
accidentally.
parent 5e6b0382
......@@ -104,9 +104,16 @@ script:
- |
if test "$BUILD_TYPE" == 'autotools'; then
git clean -fdx &&
./autogen.sh --with-systemd-logind=no --enable-more-warnings=no --enable-ifcfg-rh --enable-config-plugin-ibft --enable-ifupdown --enable-tests &&
./autogen.sh --prefix="$PWD/INST" --with-systemd-logind=no --enable-more-warnings=no --enable-ifcfg-rh --enable-config-plugin-ibft --enable-ifupdown --enable-tests &&
make -j4 &&
./contrib/travis/travis-check.sh
if [ "$CC" == gcc ]; then
sudo locale-gen de_DE.UTF-8 &&
sudo locale-gen pl_PL.UTF-8 &&
sudo make install &&
NM_TEST_CLIENT_CHECK_L10N=1 ./contrib/travis/travis-check.sh
else
./contrib/travis/travis-check.sh
fi
fi
env:
......
......@@ -3780,6 +3780,62 @@ EXTRA_DIST += \
clients/tui/meson.build \
clients/tui/newt/meson.build
###############################################################################
# clients/tests
###############################################################################
check-local-clients-tests-test-client: clients/cli/nmcli clients/tests/test-client.py
mkdir -p "$(builddir)/clients/tests/"
GI_TYPELIB_PATH="$(abs_builddir)/libnm$${GI_TYPELIB_PATH:+:$$GI_TYPELIB_PATH}" \
LD_LIBRARY_PATH="$(abs_builddir)/libnm/.libs$${LD_LIBRARY_PATH:+:$$LD_LIBRARY_PATH}" \
NM_TEST_CLIENT_BUILDDIR="$(abs_builddir)" \
NM_TEST_CLIENT_NMCLI_PATH=clients/cli/nmcli \
NM_TEST_CLIENT_IN_DBUS_SESSION=0 \
$(srcdir)/clients/tests/test-client.py -v &> "$(builddir)/clients/tests/test-client.log" && r=ok; \
cat "$(builddir)/clients/tests/test-client.log"; \
test "$$r" == ok
check_local += check-local-clients-tests-test-client
CLEANFILES += clients/tests/test-client.log
EXTRA_DIST += \
clients/tests/test-client.py \
\
clients/tests/test-client.check-on-disk/test_001-001.expected \
clients/tests/test-client.check-on-disk/test_001-002.expected \
clients/tests/test-client.check-on-disk/test_001-003.expected \
clients/tests/test-client.check-on-disk/test_001-004.expected \
clients/tests/test-client.check-on-disk/test_001-005.expected \
clients/tests/test-client.check-on-disk/test_001-006.expected \
clients/tests/test-client.check-on-disk/test_002-001.expected \
clients/tests/test-client.check-on-disk/test_002-002.expected \
clients/tests/test-client.check-on-disk/test_002-003.expected \
clients/tests/test-client.check-on-disk/test_002-004.expected \
clients/tests/test-client.check-on-disk/test_002-005.expected \
clients/tests/test-client.check-on-disk/test_002-006.expected \
clients/tests/test-client.check-on-disk/test_002-007.expected \
clients/tests/test-client.check-on-disk/test_002-008.expected \
clients/tests/test-client.check-on-disk/test_002-009.expected \
clients/tests/test-client.check-on-disk/test_002-010.expected \
clients/tests/test-client.check-on-disk/test_002-011.expected \
clients/tests/test-client.check-on-disk/test_002-012.expected \
clients/tests/test-client.check-on-disk/test_002-013.expected \
clients/tests/test-client.check-on-disk/test_002-014.expected \
clients/tests/test-client.check-on-disk/test_002-015.expected \
clients/tests/test-client.check-on-disk/test_002-016.expected \
clients/tests/test-client.check-on-disk/test_002-017.expected \
clients/tests/test-client.check-on-disk/test_003-001.expected \
clients/tests/test-client.check-on-disk/test_003-002.expected \
clients/tests/test-client.check-on-disk/test_003-003.expected \
clients/tests/test-client.check-on-disk/test_003-004.expected \
clients/tests/test-client.check-on-disk/test_003-005.expected \
clients/tests/test-client.check-on-disk/test_003-006.expected \
clients/tests/test-client.check-on-disk/test_003-007.expected \
clients/tests/test-client.check-on-disk/test_003-008.expected \
\
$(NULL)
###############################################################################
# data
###############################################################################
......
location: clients/tests/test-client.py:489:test_001()/1
cmd: $NMCLI
lang: C
returncode: 0
stdout: 277 bytes
>>>
DNS configuration:
servers: 1.2.3.4 5.6.7.8
Use "nmcli device show" to get complete information about known devices and
"nmcli connection show" to get an overview on active connection profiles.
Consult nmcli(1) and nmcli-examples(5) manual pages for complete usage details.
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:490:test_001()/2
cmd: $NMCLI
lang: pl_PL.UTF-8
returncode: 0
stdout: 310 bytes
>>>
DNS configuration:
servers: 1.2.3.4 5.6.7.8
Polecenie „nmcli device show” wyświetli pełne informacje o znanych
urządzeniach, a „nmcli connection show” wyświetli przegląd aktywnych
profili połączeń.
Strony podręcznika nmcli(1) i nmcli-examples(5) zawierają pełne informacje
o użyciu.
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:492:test_001()/3
cmd: $NMCLI -f AP -mode multiline -p d show wlan0
lang: C
returncode: 10
stdout: 0 bytes
>>>
<<<
stderr: 33 bytes
>>>
Error: Device 'wlan0' not found.
<<<
location: clients/tests/test-client.py:493:test_001()/4
cmd: $NMCLI -f AP -mode multiline -p d show wlan0
lang: de_DE.utf8
returncode: 10
stdout: 0 bytes
>>>
<<<
stderr: 47 bytes
>>>
Fehler: Gerät »wlan0« wurde nicht gefunden.
<<<
location: clients/tests/test-client.py:495:test_001()/5
cmd: $NMCLI c s
lang: C
returncode: 0
stdout: 26 bytes
>>>
NAME UUID TYPE DEVICE
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:497:test_001()/6
cmd: $NMCLI bogus s
lang: C
returncode: 2
stdout: 0 bytes
>>>
<<<
stderr: 68 bytes
>>>
Error: argument 'bogus' not understood. Try passing --help instead.
<<<
location: clients/tests/test-client.py:502:test_002()/1
cmd: $NMCLI d
lang: C
returncode: 0
stdout: 215 bytes
>>>
DEVICE TYPE STATE CONNECTION
eth0 ethernet unavailable --
wlan0 wifi unavailable --
wlan1 wifi unavailable --
wlan1 wifi unavailable --
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:504:test_002()/2
cmd: $NMCLI -f all d
lang: C
returncode: 0
stdout: 530 bytes
>>>
DEVICE TYPE STATE DBUS-PATH CONNECTION CON-UUID CON-PATH
eth0 ethernet unavailable /org/freedesktop/NetworkManager/Devices/1 -- -- --
wlan0 wifi unavailable /org/freedesktop/NetworkManager/Devices/2 -- -- --
wlan1 wifi unavailable /org/freedesktop/NetworkManager/Devices/3 -- -- --
wlan1 wifi unavailable /org/freedesktop/NetworkManager/Devices/4 -- -- --
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:506:test_002()/3
cmd: $NMCLI
lang: C
returncode: 0
stdout: 551 bytes
>>>
eth0: unavailable
"eth0"
ethernet (virtual), 72:41:AB:90:41:5D, hw
wlan0: unavailable
"wlan0"
wifi (virtual), 5A:88:5E:B6:90:40, hw
wlan1: unavailable
"wlan1"
wifi (virtual), 7C:D4:69:31:67:0B, hw
wlan1: unavailable
"wlan1"
wifi (virtual), 41:21:6B:F3:C9:4A, hw
DNS configuration:
servers: 1.2.3.4 5.6.7.8
Use "nmcli device show" to get complete information about known devices and
"nmcli connection show" to get an overview on active connection profiles.
Consult nmcli(1) and nmcli-examples(5) manual pages for complete usage details.
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:508:test_002()/4
cmd: $NMCLI -f AP -mode multiline d show wlan0
lang: C
returncode: 0
stdout: 1107 bytes
>>>
AP[1].IN-USE:
AP[1].SSID: wlan0-ap-3
AP[1].MODE: Infra
AP[1].CHAN: 1
AP[1].RATE: 54 Mbit/s
AP[1].SIGNAL: 61
AP[1].BARS: ***
AP[1].SECURITY: WPA1 WPA2
AP[2].IN-USE:
AP[2].SSID: wlan0-ap-1
AP[2].MODE: Infra
AP[2].CHAN: 1
AP[2].RATE: 54 Mbit/s
AP[2].SIGNAL: 34
AP[2].BARS: **
AP[2].SECURITY: WPA1 WPA2
AP[3].IN-USE:
AP[3].SSID: wlan0-ap-2
AP[3].MODE: Infra
AP[3].CHAN: 1
AP[3].RATE: 54 Mbit/s
AP[3].SIGNAL: 29
AP[3].BARS: *
AP[3].SECURITY: WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:509:test_002()/5
cmd: $NMCLI -f AP -mode multiline -p d show wlan0
lang: C
returncode: 0
stdout: 1558 bytes
>>>
===============================================================================
Device details (wlan0)
===============================================================================
AP[1].IN-USE:
AP[1].SSID: wlan0-ap-3
AP[1].MODE: Infra
AP[1].CHAN: 1
AP[1].RATE: 54 Mbit/s
AP[1].SIGNAL: 61
AP[1].BARS: ***
AP[1].SECURITY: WPA1 WPA2
-------------------------------------------------------------------------------
AP[2].IN-USE:
AP[2].SSID: wlan0-ap-1
AP[2].MODE: Infra
AP[2].CHAN: 1
AP[2].RATE: 54 Mbit/s
AP[2].SIGNAL: 34
AP[2].BARS: **
AP[2].SECURITY: WPA1 WPA2
-------------------------------------------------------------------------------
AP[3].IN-USE:
AP[3].SSID: wlan0-ap-2
AP[3].MODE: Infra
AP[3].CHAN: 1
AP[3].RATE: 54 Mbit/s
AP[3].SIGNAL: 29
AP[3].BARS: *
AP[3].SECURITY: WPA1 WPA2
-------------------------------------------------------------------------------
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:510:test_002()/6
cmd: $NMCLI -f AP -mode multiline -t d show wlan0
lang: C
returncode: 0
stdout: 435 bytes
>>>
AP[1].IN-USE:
AP[1].SSID:wlan0-ap-3
AP[1].MODE:Infra
AP[1].CHAN:1
AP[1].RATE:54 Mbit/s
AP[1].SIGNAL:61
AP[1].BARS:***
AP[1].SECURITY:WPA1 WPA2
AP[2].IN-USE:
AP[2].SSID:wlan0-ap-1
AP[2].MODE:Infra
AP[2].CHAN:1
AP[2].RATE:54 Mbit/s
AP[2].SIGNAL:34
AP[2].BARS:**
AP[2].SECURITY:WPA1 WPA2
AP[3].IN-USE:
AP[3].SSID:wlan0-ap-2
AP[3].MODE:Infra
AP[3].CHAN:1
AP[3].RATE:54 Mbit/s
AP[3].SIGNAL:29
AP[3].BARS:*
AP[3].SECURITY:WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:511:test_002()/7
cmd: $NMCLI -f AP -mode tabular d show wlan0
lang: C
returncode: 0
stdout: 304 bytes
>>>
NAME IN-USE SSID MODE CHAN RATE SIGNAL BARS SECURITY
AP[1] wlan0-ap-3 Infra 1 54 Mbit/s 61 *** WPA1 WPA2
AP[2] wlan0-ap-1 Infra 1 54 Mbit/s 34 ** WPA1 WPA2
AP[3] wlan0-ap-2 Infra 1 54 Mbit/s 29 * WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:512:test_002()/8
cmd: $NMCLI -f AP -mode tabular -p d show wlan0
lang: C
returncode: 0
stdout: 460 bytes
>>>
==========================
Device details (wlan0)
==========================
NAME IN-USE SSID MODE CHAN RATE SIGNAL BARS SECURITY
----------------------------------------------------------------------------
AP[1] wlan0-ap-3 Infra 1 54 Mbit/s 61 *** WPA1 WPA2
AP[2] wlan0-ap-1 Infra 1 54 Mbit/s 34 ** WPA1 WPA2
AP[3] wlan0-ap-2 Infra 1 54 Mbit/s 29 * WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:513:test_002()/9
cmd: $NMCLI -f AP -mode tabular -t d show wlan0
lang: C
returncode: 0
stdout: 165 bytes
>>>
AP[1]: :wlan0-ap-3:Infra:1:54 Mbit/s:61:*** :WPA1 WPA2
AP[2]: :wlan0-ap-1:Infra:1:54 Mbit/s:34:** :WPA1 WPA2
AP[3]: :wlan0-ap-2:Infra:1:54 Mbit/s:29:* :WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:515:test_002()/10
cmd: $NMCLI -f AP -mode multiline d show wlan0
lang: pl_PL.UTF-8
returncode: 0
stdout: 1134 bytes
>>>
AP[1].IN-USE:
AP[1].SSID: wlan0-ap-3
AP[1].MODE: Infrastruktura
AP[1].CHAN: 1
AP[1].RATE: 54 Mb/s
AP[1].SIGNAL: 61
AP[1].BARS: ***
AP[1].SECURITY: WPA1 WPA2
AP[2].IN-USE:
AP[2].SSID: wlan0-ap-1
AP[2].MODE: Infrastruktura
AP[2].CHAN: 1
AP[2].RATE: 54 Mb/s
AP[2].SIGNAL: 34
AP[2].BARS: **
AP[2].SECURITY: WPA1 WPA2
AP[3].IN-USE:
AP[3].SSID: wlan0-ap-2
AP[3].MODE: Infrastruktura
AP[3].CHAN: 1
AP[3].RATE: 54 Mb/s
AP[3].SIGNAL: 29
AP[3].BARS: *
AP[3].SECURITY: WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:516:test_002()/11
cmd: $NMCLI -f AP -mode multiline -p d show wlan0
lang: pl_PL.UTF-8
returncode: 0
stdout: 1592 bytes
>>>
===============================================================================
Informacje o urządzeniu (wlan0)
===============================================================================
AP[1].IN-USE:
AP[1].SSID: wlan0-ap-3
AP[1].MODE: Infrastruktura
AP[1].CHAN: 1
AP[1].RATE: 54 Mb/s
AP[1].SIGNAL: 61
AP[1].BARS: ***
AP[1].SECURITY: WPA1 WPA2
-------------------------------------------------------------------------------
AP[2].IN-USE:
AP[2].SSID: wlan0-ap-1
AP[2].MODE: Infrastruktura
AP[2].CHAN: 1
AP[2].RATE: 54 Mb/s
AP[2].SIGNAL: 34
AP[2].BARS: **
AP[2].SECURITY: WPA1 WPA2
-------------------------------------------------------------------------------
AP[3].IN-USE:
AP[3].SSID: wlan0-ap-2
AP[3].MODE: Infrastruktura
AP[3].CHAN: 1
AP[3].RATE: 54 Mb/s
AP[3].SIGNAL: 29
AP[3].BARS: *
AP[3].SECURITY: WPA1 WPA2
-------------------------------------------------------------------------------
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:517:test_002()/12
cmd: $NMCLI -f AP -mode multiline -t d show wlan0
lang: pl_PL.UTF-8
returncode: 0
stdout: 462 bytes
>>>
AP[1].IN-USE:
AP[1].SSID:wlan0-ap-3
AP[1].MODE:Infrastruktura
AP[1].CHAN:1
AP[1].RATE:54 Mb/s
AP[1].SIGNAL:61
AP[1].BARS:***
AP[1].SECURITY:WPA1 WPA2
AP[2].IN-USE:
AP[2].SSID:wlan0-ap-1
AP[2].MODE:Infrastruktura
AP[2].CHAN:1
AP[2].RATE:54 Mb/s
AP[2].SIGNAL:34
AP[2].BARS:**
AP[2].SECURITY:WPA1 WPA2
AP[3].IN-USE:
AP[3].SSID:wlan0-ap-2
AP[3].MODE:Infrastruktura
AP[3].CHAN:1
AP[3].RATE:54 Mb/s
AP[3].SIGNAL:29
AP[3].BARS:*
AP[3].SECURITY:WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:518:test_002()/13
cmd: $NMCLI -f AP -mode tabular d show wlan0
lang: pl_PL.UTF-8
returncode: 0
stdout: 338 bytes
>>>
NAME IN-USE SSID MODE CHAN RATE SIGNAL BARS SECURITY
AP[1] wlan0-ap-3 Infrastruktura 1 54 Mb/s 61 *** WPA1 WPA2
AP[2] wlan0-ap-1 Infrastruktura 1 54 Mb/s 34 ** WPA1 WPA2
AP[3] wlan0-ap-2 Infrastruktura 1 54 Mb/s 29 * WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:519:test_002()/14
cmd: $NMCLI -f AP -mode tabular -p d show wlan0
lang: pl_PL.UTF-8
returncode: 0
stdout: 530 bytes
>>>
===================================
Informacje o urządzeniu (wlan0)
===================================
NAME IN-USE SSID MODE CHAN RATE SIGNAL BARS SECURITY
-----------------------------------------------------------------------------------
AP[1] wlan0-ap-3 Infrastruktura 1 54 Mb/s 61 *** WPA1 WPA2
AP[2] wlan0-ap-1 Infrastruktura 1 54 Mb/s 34 ** WPA1 WPA2
AP[3] wlan0-ap-2 Infrastruktura 1 54 Mb/s 29 * WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:520:test_002()/15
cmd: $NMCLI -f AP -mode tabular -t d show wlan0
lang: pl_PL.UTF-8
returncode: 0
stdout: 192 bytes
>>>
AP[1]: :wlan0-ap-3:Infrastruktura:1:54 Mb/s:61:*** :WPA1 WPA2
AP[2]: :wlan0-ap-1:Infrastruktura:1:54 Mb/s:34:** :WPA1 WPA2
AP[3]: :wlan0-ap-2:Infrastruktura:1:54 Mb/s:29:* :WPA1 WPA2
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:522:test_002()/16
cmd: $NMCLI c
lang: C
returncode: 0
stdout: 126 bytes
>>>
NAME UUID TYPE DEVICE
con-1 5fcfd6d7-1e63-3332-8826-a7eda103792d ethernet --
<<<
stderr: 0 bytes
>>>
<<<
location: clients/tests/test-client.py:524:test_002()/17
cmd: $NMCLI c s con-1
lang: C
returncode: 0
stdout: 990 bytes
>>>
connection.id: con-1
connection.uuid: 5fcfd6d7-1e63-3332-8826-a7eda103792d
connection.stable-id: --
connection.type: 802-3-ethernet
connection.interface-name: --
connection.autoconnect: yes
connection.autoconnect-priority: 0
connection.autoconnect-retries: -1 (default)
connection.auth-retries: -1
connection.timestamp: 0
connection.read-only: no