Skip to content

[th/cloud-setup-fix-containers] better handle other route configuration (including containers)

Thomas Haller requested to merge th/cloud-setup-fix-containers into main

nm-cloud-setup is supposed to automatically configured the network in the cloud environment. As such, when a user wants a special network configuration, then it seems reasonable and expected that they disable the automatism -- if it doesn't do what they want.

Still, the automatism needs to work well in common cases. In particular, in cases where the user doesn't do something special. Such a case is running containers. The container runtime might create another interface and setup routes in the main table.

With the current setup of having

        0:  from all lookup local
        30400:  from 10.0.10.5 lookup 30400
        32766:  from all lookup main
        32767:  from all lookup default

and

        default via 10.0.10.1 dev eth0 table 30400 proto static metric 10
        10.0.10.1 dev eth0 table 30400 proto static scope link metric 10

it means that the "default" route hijacks all routes. That's wrong.

For issues see:

===

This branch tries to fix that in two ways:

  1. commit 'cloud-setup: skip configuring policy routing if there is only one interface/address':

    If nm-cloud-setup only detects only one interface/address, there is no reason to configure any policy routing at all. That should solve the majority of cases, because having multiple IP addresses is in fact not something that is commonly done (I claim).

  2. commit 'cloud-setup: use suppress_prefixlength rule to honor non-default-routes in the main table'

    Add a rule

30300:  from all lookup main suppress_prefixlength 0

This means to first look at the route table for any non-default routes. If found, that one is used and we skip our source-based policy routing rules. Only if the destination is only reachable via the default route, continue with source based routing (and look at tables 30400+, which has default routes configured). This effectively shortcuts the mechanism in many scenarios. Which by itself might be a problem, and maybe this make what nm-cloud-setup does useless. Dunno...

@nmeyerhans, what do you think?

Merge request reports