Skip to content

Add event annotation protocol

Peter Hutterer requested to merge wip/wayland-annotation into main

Note: this is very much still in the brainfart stage.


This adds an "event annotation" protocol. This idea originally came from @carlosg for panscrolling on tablet pens but this implementation is much more generic. The protocol is currently a mutter-private one until we can figure it if this could event work, there are quite a few cans of worms to eat. The corresponding client-side implementation PoC is gtk!1. Note: currently not compatible after some rework

The protocol has three interfaces:

  • event_annotation_manager as top-level object, responsible for creating
  • event_annotation_object as the annotation that says what to actually do, which creates
  • event_annotation on a specific input event (sequence)

So the event_annotation_object (better name needed...) describers what should be done and a short-lived event_annotation object marks the actual events to be annotated.

For added flexibility, an action has metadata specified by the mimetype and a schema to parse the data, this means we can have an external specification for metadata like this:

{
  "version": 1,
  "identifier": "org.freedesktop.Scroll",
  "type": "oneshot",
  "arguments": {
    "transformation": [a, b, c, d, e,f ]
  }
}

The identifier is a reverse domain name identifier for easy filtering (i.e. clients to ignore annotations they don't know about). Ideally, we have most toolkits implement the org.freedesktop actions eventually so we get cross-platform functionality. With configuration tools an action could then be assigned to various interactions, including the potential for per-application actions (mouse button 5 does foo in application A but bar in application B).

  • open question: how would a compositor know about about application-specific actions?

The current patchset shows a few such actions in the org.fredesktop namespace, though they're by no means final.

The protocol itself currently looks like this (excluding mimetype negotation):

  • event_annotation_manager1.annotation_object(new_id=123, name="org.freedesktop.v1.Foo")
  • event_annotation_object1@123.data(mimetype=foo, schema=bar, fd=fd)
  • event_annotation_object1@123.done()
  • event_annotation_object1@123.annotation(new_id=987)
  • event_annotation_annotation1@987.annotate() ← this means this annotation applies to the current .frame
  • event_annotation_annotation1@987.finish() ← annotations may last for multiple frames in which case finish is called after the last annotation
  • wl_pointer.button(BTN_TASK, press=true)
  • wl_pointer.frame()

The protocol is relatively trivial, most of the difficulty will be in specifying how the actions must be interpreted and implementing them on the client side.

The current patch set add some basic scaffolding and hardcodes an org.freedesktop.ScrollOnce annotation to be added to each BTN_RIGHT event. That should, ideally, trigger one mouse wheel click on the client whenever the right button is pressed.

The only real meat in this patch that's worthy of deeper investigation is the protocol XML file - all the other code is too early to contemplate in detail.

cc @carlosg, @jadahl

Edited by Peter Hutterer

Merge request reports