gbm: fail early when modifier list only contains INVALID
The current check only accomodates for a list with a single INVALID item. However the driver won't be able to pick any modifier if the list only contains INVALID. This includes the following cases:
- The modifier list is empty (count == 0)
- The modifier list contains more than a single item, but all items are INVALID
In these cases, also fail early.