RemoteDesktop: Add API to allow receiving keymaps in addition to keycodes and to override Capslock and Numlock state
In contrary to the VNC backend in g-r-d, the RDP backend doesn't send keysyms to mutter. Instead, keycodes are sent to mutter in addition to the client keyboard layout, variant and type.
Additionally, RDP informs the server about its current Capslock, Numlock, Scrolllock and Kanalock state (KBD_SYNC_SCROLL_LOCK, KBD_SYNC_NUM_LOCK, KBD_SYNC_CAPS_LOCK, KBD_SYNC_KANA_LOCK).
This implies that the keyboard handling needs to either happen in g-r-d or in mutter. Having the keyboard handling in g-r-d allows us to create keysyms in g-r-d and send them to mutter.
This however has to problems:
- Client windows on the server side won't know the actual current state of locked keys like Capslock or Numlock
- The actual input that the client windows receive is not defined: Although the current implementation in https://gitlab.gnome.org/jadahl/gnome-remote-desktop/-/merge_requests/11 sends keysyms, mutter will still modify them.
For example: if numlock is on the server side activated, but not on the client side, the client windows will still receive numbers (which was not the intention) when pressing the arrow keys on the numpad.
Additionally, if both sides have numlock activated, the actual input will be garbage, e.g. the letters A, B, C, D, E will entered when pressing the numbers 2, 4, 5, 6, 8 on the numpad.
The same applies to capslock.
To solve this situation, mutter needs to take over the keyboard handling for keycodes. Currently, there is an API for keycodes (https://gitlab.gnome.org/GNOME/mutter/-/blob/1faba58ccc4b79e945c79fef16e27403d38e6212/src/org.gnome.Mutter.RemoteDesktop.xml#L86) but it doesn't follow the keyboard -layout
, -variant
, -type
of the client, meaning the actual input is different than the intended one.
This API (that allows keycode input) should be either modified to also have the keyboard -layout, -variant, -type as parameter or a different API should be added where a remote desktop session can submit these values to mutter, where mutter saves them to use them when an keycode input event happens.
The second part is that the capslock, numlock, ... states can change when the remote desktop client is minimized or unfocused.
RDP sends an update to the server with the current capslock, numlock states when it gets focused again. mutter should offer an API to override its current kbd states to be able to properly process the remote input.
I'm not sure whether it is possible or relevant to update the "kana state" and "scroll lock state", but the API should at least allow g-r-d to submit the new capslock
and numlock
states.
These two APIs are necessary to properly process input for the RDP backend in g-r-d (https://gitlab.gnome.org/jadahl/gnome-remote-desktop/-/merge_requests/11)
I propose to have something like the following two APIs in mutter:
<method name="NotifyKeyboardKeycode">
<arg name="keycode" type="u" direction="in" />
<arg name="state" type="b" direction="in" />
<arg name="kbdtype" type="s" direction="in" />
<arg name="kbdlayout" type="s" direction="in" />
<arg name="kbdvariant" type="s" direction="in" />
</method>
<method name="NotifyKeyboardStates">
<arg name="capslock" type="b" direction="in" />
<arg name="numlock" type="b" direction="in" />
</method>
With these two APIs mutter can switch to the remote kbd keymap when a remote input arrives, process the keycode and switch back to the local kbd keymap.