NM in initrd someway ignores /etc/NetworkManager/system-connections/
I encountered this issue when I copied the network configurations files in the 1st kernel to the kdump initrd's /etc/NetworkManager/system-connections
directly and tried to make NM bring up the connections specified in /etc/NetworkManager/system-connections
.
I set up a bonded network with the following commands,
nmcli con add type bond ifname mybond0
nmcli con add type ethernet ifname eth0 master mybond0
nmcli c up bond-slave-eth0
and also changed kexec-tools so the following cmdline will be obtained by dracut,
CMDLINE=' rd.neednet kdumpnic=mybond0 ifname=kdump-eth0:52:54:00:12:34:59 elfcorehdr=0xaf000000 BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.13.15-200.fc34.x86_64 ro no_timer_check net.ifnames=0 console=tty1 console=ttyS0,115200n8 irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 acpi_no_memhotplug transparent_hugepage=never nokaslr hest_disable novmcoredd cma=0 hugetlb_cma=0 rd.debug disable_cpu_apicid=0 trace_buf_size=1'
There are two variants of this issue.
nm-initrd-generator called by dracut
Full console log with rd.debug
: console_log_nm_initrd_generator_called.log
NM would generate a default connection and bring up this connection. The copied connections in /etc/NetworkManager/system-connections
would be ignored,
<trace> [1631714108.7080] settings: storage[66957282-a5d0-48a2-aa12-c509e11505fa,6e071a03366ac8a2/keyfile]: change event with connection "Wired Connection" (file "/run/NetworkManager/system-connections/default_connection.nmconnection")
<trace> [1631714108.7102] settings: storage[f627e3fb-749e-4bf0-94f2-1dad67768c38,ffcf79439531196e/keyfile]: change event with connection "bond-slave-eth0" (file "/etc/NetworkManager/system-connections/bond-slave-eth0.nmconnection")
<trace> [1631714108.7115] settings: storage[bc617e35-f0f7-4448-926d-76fc509e270f,9bf7a3bffb3f84a3/keyfile]: change event with connection "bond-mybond0" (file "/etc/NetworkManager/system-connections/bond-mybond0.nmconnection")
<trace> [1631714108.7128] ifcfg-rh: Could not read directory '/etc/sysconfig/network-scripts': Error opening directory ?/etc/sysconfig/network-scripts?: No such file or directory
<trace> [1631714108.7139] settings: update[66957282-a5d0-48a2-aa12-c509e11505fa]: adding connection "Wired Connection" (6e071a03366ac8a2/keyfile)
<trace> [1631714108.7153] settings-connection[0e21b5adbea5a389,66957282-a5d0-48a2-aa12-c509e11505fa]: update system secrets: secrets set
<trace> [1631714108.7163] settings-connection[0e21b5adbea5a389,66957282-a5d0-48a2-aa12-c509e11505fa]: update agent secrets: secrets set
<trace> [1631714108.7170] settings-connection[0e21b5adbea5a389,66957282-a5d0-48a2-aa12-c509e11505fa]: timestamp: no timestamp from keyfile database "/var/lib/NetworkManager/timestamps"
<trace> [1631714108.7180] settings-connection[0e21b5adbea5a389,66957282-a5d0-48a2-aa12-c509e11505fa]: no seen-bssids from keyfile database "/var/lib/NetworkManager/seen-bssids"
<trace> [1631714108.7189] settings-connection[0e21b5adbea5a389,66957282-a5d0-48a2-aa12-c509e11505fa]: update settings-connection flags to unsaved,visible (was none)
<trace> [1631714108.7198] dbus-object[0e21b5adbea5a389]: export: "/org/freedesktop/NetworkManager/Settings/1"
<debug> [1631714108.7207] ++ connection 'new connection' (0x5586a8394400/NMSimpleConnection/"802-3-ethernet"): [/org/freedesktop/NetworkManager/Settings/1]
<debug> [1631714108.7220] ++ connection [ 0x7f5578003de0 ]
<debug> [1631714108.7226] ++ connection.autoconnect-retries = 1
<debug> [1631714108.7230] ++ connection.id = 'Wired Connection'
<debug> [1631714108.7235] ++ connection.multi-connect = 3
<debug> [1631714108.7239] ++ connection.permissions = []
<debug> [1631714108.7243] ++ connection.type = '802-3-ethernet'
<debug> [1631714108.7247] ++ connection.uuid = '66957282-a5d0-48a2-aa12-c509e11505fa'
<debug> [1631714108.7253] ++ connection.wait-device-timeout = 60000
<debug> [1631714108.7259] ++ 802-3-ethernet [ 0x5586a8341150 ]
<debug> [1631714108.7265] ++ 802-3-ethernet.mac-address-blacklist = []
<debug> [1631714108.7270] ++ 802-3-ethernet.s390-options = ((GHashTable*) 0x5586a83930c0)
<debug> [1631714108.7279] ++ ipv4 [ 0x5586a83a4110 ]
[ 2.637279] systemd[1]: Started D-Bus System Message Bus.
<debug> [1631714108.7288] ++ ipv4.addresses = ((GPtrArray*) 0x5586a83942c0)
<debug> [1631714108.7295] ++ ipv4.dhcp-timeout = 90
<debug> [1631714108.7300] ++ ipv4.dns = []
<debug> [1631714108.7304] ++ ipv4.dns-search = []
<debug> [1631714108.7309] ++ ipv4.method = 'auto'
<debug> [1631714108.7313] ++ ipv4.routes = ((GPtrArray*) 0x5586a8369e00)
<debug> [1631714108.7318] ++ ipv4.routing-rules = <unknown>
<debug> [1631714108.7322] ++ ipv6 [ 0x5586a83a41e0 ]
<debug> [1631714108.7327] ++ ipv6.addr-gen-mode = 0
<debug> [1631714108.7331] ++ ipv6.addresses = ((GPtrArray*) 0x5586a8394500)
<debug> [1631714108.7336] ++ ipv6.dhcp-timeout = 90
<debug> [1631714108.7340] ++ ipv6.dns = []
[ 2.643342] NetworkManager[287]: <info> [1631714108.6134] bus-manager: acquired D-Bus service "org.freedesktop.NetworkManager"
<debug> [1631714108.7351] ++ ipv6.dns-search = []
<debug> [1631714108.7357] ++ ipv6.method = 'auto'
<debug> [1631714108.7362] ++ ipv6.routes = ((GPtrArray*) 0x5586a83948c0)
<debug> [1631714108.7372] ++ ipv6.routing-rules = <unknown>
The default connection was first generated by nm-initrd-generator. Later, in the emergency shell, after I even deleted /run/NetworkManager/system-connections/default_connection.nmconnect and "/usr/sbin/NetworkManager --configure-and-quit=initrd --no-daemon" could generate a connection file. If I made dracut to run nm-initrd-generator with -s
, the behaviour of nm was the same to what's below.
nm-initrd-generator not called by dracut
Full console log with rd.debug
: console_log_nm_initrd_generator_not_called.log
NetworkManager would log "startup complete" before bringing up connections in specified in etc/NetworkManager/system-connections. This causes nm-wait-online-initrd.service to quit early and leads to remote fs failed to be mounted.
Btw, if I deliberately make dracut wait for 10s, the connections could be brought up. Or if I run /usr/sbin/NetworkManager --configure-and-quit=initrd --no-daemon
in the emergency shell, the connections could also be brought up.