This prevents issues from happening when processing Enter/Leave events
while in another kind of grab op like a Wayland popup or resizing a
window.

This can't ever really happen except outside of a race condition,
with the X server, since we won't ever pass input events to the
X server in any of these cases, but it can't hurt to be more correct
about what the intended operation is.