[WAYLAND] Need fix or new API for generated mouse clicks to work.
When Wayland is being used, AtspiComponent returns the same position for objects regardless of whether DESKTOP_COORDS or WINDOW_COORDS are being specified. And the coordinates we get are with respect to the current window. However, generate_mouse_event expects absolute coordinates.
This is problematic because Orca is asking for the absolute/desktop coordinates and then using those coordinates when calling generate_mouse_event to synthesize clicks. As a result, Orca is trying to click on one thing, but is actually clicking on something else. A more specific example:
- Maximize Thunderbird
- Place the gtk3-demo window on top of the Thunderbird window, but in the middle of your screen
- Arrow in the tree of demos
- Use KP_Divide to synthesize the click
When I do this using Wayland, the click is synthesized on the Thunderbird window underneath gtk3-demo for the reasons stated in the first paragraph of this report.
One solution (which I'd prefer) is that asking for the absolute/desktop coordinates would cause the absolute/desktop coordinates to be returned.
Another solution would be to add new API to AT-SPI2 so that one could specify the window in which the mouse click should take place.
I'm open to additional suggestions.
In the meantime, I'll see if I can work around this via AtspiAction. But that won't work reliably for all things (what if there is no appropriate or recognized action?). Thus it would be great to fix this within AtspiComponent or the mouse event synthesis code.