For IP addresses, kernel cares about the order in which they were added. This mostly affects source address selection, and the "secondary" flag for IPv4 addresses. The order is thus related to the priority of an address.
There is no direct kernel API to change the order. Instead, we have to add them in the correct order. During a sync, if an address already exists in the wrong order, we need to remove it, and re-add it.
For IPv6, addresses added last, will be more important. That means, if we want to add a less-important address, we need to remove the more important one, if it's already configured.
Previously, we would first iterate over all addresses and remove those that had a conflicting order. This means, that we would potentially remove all addresses for a short time, before readding them. That seems problematic.
Instead, first track all IPv6 addresses that are in the wrong order. And in the step when we add/update the address, remove it. This means, we don't remove all addresses and re-add all, instead, we only remove and re-add it one address at a time. This way the time for which the address on the interface is missing is shorter. More importantly, we will never remove all addresses at the same time.
TODO: I think the same could be done for IPv4 addresses, although there the first address becomes the primary.
EDIT: in the meantime, I did the same for IPv4 and reworded the commit message. For an up-to-date description, read the commit message on the merge-request.