55773 files in ~/.cache/fontconfig/

It looks weird. Some of the files are 6 years old. I'm not sure if this cache is effective at all, as most of large caches I have seen use prefixing to speedup filesystem lookup.

root@localhost:~# echo 3 > /proc/sys/vm/drop_caches 
root@localhost:~# time ls -l /home/bodqhrohro/.cache/fontconfig/ > /dev/null

real	0m25,959s
user	0m0,964s
sys	0m2,015s

Moreover, it's still there after I ran fc-cache -frs (from user, of course), with the same timestamps. Would it be safe if I just clear the directory? Or all of these thousands of files will reappear anyway on the next cache rebuild?