platform: support IPv6 mulitpath routes and fix cache inconsistency
Add support for IPv6 multipath routes, by treating them as single-hop routes. Otherwise, we can easily end up with an inconsistent platform cache. Background: ----------- Routes are hard. We have NMPlatform which is a cache of netlink objects. That means, we have a hash table and we cache objects based on some identity (nmp_object_id_equal()). So those objects must have some immutable, indistinguishable properties that determine whether an object is the same or a different one. For routes and routing rules, this identifying property is basically a subset of the attributes (but not all!). That makes it very hard, because tomorrow kernel could add an attribute that becomes part of the identity, and NetworkManager wouldn't recognize it, resulting in cache inconsistency by wrongly thinking two different routes are one and the same. Anyway. The other point is that we rely on netlink events to maintain the cache. So when we receive a RTM_NEWROUTE we add the object to the cache, and delete it upon RTM_DELROUTE. When you do `ip route replace`, kernel might replace a (different!) route, but only send one RTM_NEWROUTE message. We handle that by somehow finding the route that was replaced/deleted. It's ugly. Did I say, that routes are hard? Also, for IPv4 routes, multipath attributes are just a part of the routes identity. That is, you add two different routes that only differ by their multipath list, and then kernel does as you would expect. NetworkManager does not support IPv4 multihop routes and just ignores them. Also, a multipath route can have next hops on different interfaces, which goes against our current assumption, that an NMPlatformIP4Route has an interface (or no interface, in case of blackhole routes). That makes it hard to meaningfully support IPv4 routes. But we probably don't have to, because we can just pretend that such routes don't exist and our cache stays consistent (at least, until somebody calls `ip route replace` *sigh*). Not so for IPv6. When you add (`ip route append`) an IPv6 route that is identical to an existing route -- except their multipath attribute -- then it behaves as if the existing route was modified and the result is the merged route with more next-hops. Note that in this case kernel will only send a RTM_NEWROUTE message with the full multipath list. If we would treat the multipath list as part of the route's identity, this would be as if kernel deleted one routes and created a different one (the merged one), but only sending one notification. That's a bit similar to what happens during `ip route replace`, but it would be nightmare to find out which route was thereby replaced. Likewise, when you delete a route, then kernel will "subtract" the next-hop and sent a RTM_DELROUTE notification only about the next-hop that was deleted. To handle that, you would have to find the full multihop route, and replace it with the remainder after the subtraction. NetworkManager so far ignored IPv6 routes with more than one next-hop, this means you can start with one single-hop route (that NetworkManger sees and has in the platform cache). Then you create a similar route (only differing by the next-hop). Kernel will merge the routes, but not notify NetworkManager that the single-hop route is not longer a single-hop route. This can easily cause a cache inconsistency and subtle bugs. For IPv6 we MUST handle multihop routes. Kernels behavior makes little sense, if you expect that routes have an immutable identity and want to get notifications about addition/removal. We can however make sense by it by pretending that all IPv6 routes are single-hop! With only the twist that a single RTM_NEWROUTE notification might notify about multiple routes at the same time. This is what the patch does. The Patch --------- Now one RTM_NEWROUTE message can contain multiple IPv6 routes (NMPObject). That would mean that nmp_object_new_from_nl() needs to return a list of objects. But it's not implemented that way. Instead, we still call nmp_object_new_from_nl(), and the parsing code can indicate that there is something more, indicating the caller to call nmp_object_new_from_nl() again in a loop to fetch more objects. In practice, I think all RTM_DELROUTE messages for IPv6 routes are single-hop. Still, we implement it to handle also multi-hop messages the same way. Note that we just parse the netlink message again from scratch. The alternative would be to parse the first object once, and then clone the object and only update the next-hop. That would be more efficient, but probably harder to understand/implement. https://bugzilla.redhat.com/show_bug.cgi?id=1837254#c20 (cherry picked from commit dac12a8d) (cherry picked from commit 698cf109)