wayland: Clean up xwayland grabs even if surface is gone
If the surface is gone before
called, we would bail out early leaving a empty grab, which will later
cause a segfault as soon as a key is pressed later on.
Make sure we clean up the keyboard grab even if the surface is gone.
Closes: #255 (closed)