wl_output modes cannot be removed
The kernel might decide to prune a connector's mode, e.g. when the link-status becomes bad. We have no way to send this new information to clients.
In the first place it's not clear how modes updates are supposed to happen. Should the client assume that all supported modes are sent when binding, and that all subsequent updates are not new modes but can only change the current mode? How should virtual outputs behave when resized?
(Finally, it's not clear what modes are useful for, especially non-current modes. The fullscreen-shell still uses those for modeset, but IMHO this isn't enough to have modes in the core protocol)