If we were cancelled, it could mean we teared down, meaning fetching
manager instances will attempt to fetch past freed instances. Handle
this by waiting with the fetching until we know we weren't cancelled.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2140>