clover: move tokenize function to algorithm

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <!4974>
31 jobs for !4974 with clover/going_cl1.2 in 4 minutes and 55 seconds (queued for 5 seconds)
merge request