Move the const_cast from assignment to free
The problem is that some of the times the pointers hold values to const tables and some others hold values to dynamic memory.
Instead of const_casting the const tables when needed and holding a non const pointer, we hold a const pointer and only const cast on free if needed