gdk3: interpolate scroll events to screen refresh, part 4 of 4
TLDR
The issue description was taken from #2025.
Scrolling a window by using a touchpad is jittery even when we move the fingers at a constant speed. The somewhat counterintuitive notion here is that the scroll is jittery even though the input event rate is higher than the screen refresh rate. In fact even if the whole stack was fully optimized this issue would still exist.
The root cause
The root cause of the problem is that the input events are not synchronized to the screen refresh - they operate at different rates. As an example, the Apple external "Magic" trackpads emit input events at a rate of about 90Hz. A standard display operates at 60Hz. In this setup we get 2 input events in the first display frame, 1 input event in the second display frame, 2 in the third, 1 in the forth etc.
If, for example, each scroll event results in a movement of 4 pixels, we'll get 8px on the first frame (2 events of 4px each), 4px on the second frame (1 event of 4px), 8px on the third frame, 4px on the forth etc. So the window is scrolled by a wildly different number of pixels in each frame, even though the fingers moved on the touchpad at a constant speed.
The fix
The original MR was split to !1559, !1560, !1561 and this one.
This is the fourth part in an MR series aimed at solving jitter in animation based on continuous input. See #2025 and #2573 for details about the issues, and below for a general description of the solution. With this MR in place, interpolation of smooth scroll events is fully handled.
Depends on, and includes, !1561
Continued in
What it does
Keep a history of smooth scroll events, and use linear interpolation to estimate the position of the input device at frame display time rather than at the original event time. When interpolation is enabled, the original input events are not sent to the listeners. Instead, newly synthesized events are sent, once per display frame.
More details
With this MR applied scrolling is visibly smoother when using, for example, an external Apple touchpad (issue #2025, see also the attached videos). In addition, by using frame update callbacks, input events are synthesized even for frames which did not originally receive an input event (issue #2573). This should result in smoother scroll animation on setups combining a standard 125Hz input device with high refresh rate monitors such as those capable of 144Hz or even 240Hz. Furthermore since only a single event is sent per display frame, regardless of the input device event frequency, it has the potential of reducing cpu usage.
The implementation is hooked to GdkWindow in order to benefit any GTK widget as well as other GDK-based toolkits. It was tested with gtk3-demo, gedit, Epiphany and Firefox, both on Wayland, XWayland and X11 sessions.
Scroll before: scroll_jitter
Scroll after: scroll_smooth
Dependencies and prerequisites
A kernel regression from about v5.4.0 causes timestamps of input events to be jittery for certain devices. I filed a bug report here. If you use a kernel of a similar or newer version, you might be affected. Please see the bug report for details. Affected setups won't see the benefit of this MR. A fix was released for kernel v5.8 and above, hopefully it will be backported to the stable kernels soon. -- update (2020-8-8): the fix was committed to the upstream stable kernels : v5.4.53 and v5.7.10.
The Apple external "Magic" trackpad, 2nd generation, had the wrong touch thresholds applied, resulting in lost touchpoints and problematic behavior. This was fixed in libinput v1.15.6. If you are using an older version you'll have to backport the fix.
This MR used to depend on a few frame-clock related fixes, however this is now solved with the recent merge of !1931 (merged).
In addition it is recommended to apply something like !1117 (comment 636494) to libinput since the default touchpad scroll speed is not ideal - see #1308 (closed).
Corresponding bugs and MRs
This is an implementation similar to !1111 but for GDK 3, and is more up-to-date. It is also tested with real world applications, so I'd rather land it first then update the GDK 4 code accordingly.