Trying to get the xwindow of a wayland only window would fail when casting to a x11 window
Fixes: bc9cd123 ("window: Move xwindow to WindowX11")
cc @rmader, this should fix the flaky CI