When we get a buffer from the server, mark it busy, when we queue it
again, mark it unbusy.
Refuse to dequeue a buffer when it is busy.