This fixes an problem left in f682e056 due to an incorrect cherry-pick (my bad, I've ended up testing a wrong thing :-( )
We must use old listener->listener to deliver the touch event. Otherwise grab won't let the event through and the abovementioned commit has no effect.
cc @whot