It's possible for a released pointer to have repick / set_focus on it as
part of sync_input_focus. When the pointer is actually re-init'd, it
will memset 0, which can cause corruption as our destroy listener has
already been added.
Released devices should be idempotent, so just make sure method calls on
them don't have any effect.