Skip to content

Draft: Mutter SDK

Jonas Ådahl requested to merge jadahl/mutter:wip/sdk into main

This merge request adds a new way of running and interacting with mutter (and gnome-shell) by adding two things:

  • A "viewer" that connects to the mutter instance of the session and creates a virtual monitor which it then sends input to.
  • A "sdk" argument to mutter that launches this viewer when one passes --sdk to mutter or gnome-shell.

The viewer currently only uses the remote desktop and screen cast APIs and require nothing else, except an addition to the screen cast API where one can add new monitors without starting new sessions. What would probably be useful, however, is an org.gnome.Mutter.Sdk D-Bus API which one can use to control certain things. A MetaSdk object is introduced which is instantiated when passing --sdk which can probably learn more things, e.g. introspecting Wayland/X11 displays, and doing other things.

The viewer needs some improvement, for example:

  • Virtual monitor screen cast streams always means an extra copy - that should easily be avoided in mutter
  • The cursor is embedded in the stream. I have the cursor sprite code from https://gitlab.gnome.org/feaneron/pipewire-media-stream/ (which parts of the PipeWire stuff were based on) so should probably be easy to add.
  • There is no Vsync yet. See https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/1484.
  • Input is incomplete. Done so far:
    • Pointer motion events
    • Pointer button events
    • Pointer scroll events
    • Keyboard key events
    • Touch events
    • Tablet events (Not going to bother until there is something like libeis in place)
  • Make it possible to inhibit keyboard shortcuts in the host environment

Demo: Screencast_from_2021-07-31_23_22_02

Edited by Jonas Ådahl

Merge request reports