si-adapter: handle dont-remix streams

Don't-remix streams don't change their channel mapping based on
the device they connect to but are always configured with their
default channel mapping, just like devices.

This fixes the gnome-control-center channel test.

This patch also removes the 1 channel -> many patch because it is
not the right thing to do in this case and should be implemented
with some proper channel mapping patch later.