user: Remove user from extra_admin_groups when demoting them from admin

Filter out the extra_admin_groups from the group list when a user is no longer an admin.

Signed-off-by: Philip Withnall withnall@endlessm.com