Mark a keyboard inactive before deactivating to avoid races
Mark the keyboard we're deactivating (removing watches for input) just before actually removing the watches. This avoids running into a case where we might try to remove the watch again in parallel from this, crashing if this happens while we're freeing objects too.