xwayland: Handle shutting down without having started
We initialize, but might not start, e.g. when a test case just needs a backend and doesn't start mutter. When cleaning up, we'll still try to clean up Xwayland integration, and this commit handles cleaning up without having made the mess.
Follow up from !1822 (merged). CI missed it because it doesn't have new enough libX11.