We don't want to fallback for 'random' touch sequences, since
that could lead to all kinds of pairedness and other violations.
Since the X server already tells us what touch events it would
have used for emulating pointer events, we just use that information
here.