Previously, the fence readers array was only cleared when there are no unsignaled fences, which almost never happens in some situations.
That resulted in the array having thousands of NULL pointers, but only a handful of active readers.
With this patch, all the unsignaled readers are moved to the front of the array, effectively building a new array only containing the active readers in-place. This results in the readers array usually only having a couple of elements.
Reallocation errors are not handled, because it is impossible for the array to grow -- we aren't getting new readers out of /dev/null.