Skip to content

RDP: Add support for virtual monitors + enhance hardware acceleration support

This MR implements support for virtual monitors, frame tracking, and it enhances the hardware acceleration support by eliminating latency in various places and moving operations out of the graphics thread.

The reason why the hardware acceleration stuff is in this MR is to ensure that hardware acceleration can still work fine, even with on-the-fly monitor resolution changes, especially when using DMA-bufs.

Since registering GL resources for the usage of external APIs (like CUDA) takes a bit of time, frames are now handled inside frame pools. These frames have preallocated resources and are being reused (mapping and unmapping these resources is fast).

Damage detection now runs on the GPU, if possible, as it is in almost all cases much faster, leading to reduced frame latencies.

Uploading frame content is faster with glBufferSubData, than using CUDA APIs. This will benefit the NVENC path (lower latency + since this uploading operation does not happen in the graphics thread any more, the encoding operation takes less time, which allows reaching higher framerates (framerates are still dependent on the client side, i.e. if the client side is too slow, g-r-d still has to throttle the encoding rate)).

While the NVIDIA driver for my Kepler GPU (GTX 660) does not allow the usage of DMA-bufs, I still implemented a way to use DMA-bufs with hardware acceleration.
The result is that no raw frame data transfer happens between the host and the device (GPU).
Both DMA-buf path and non-DMA-buf path are very similar, so it should work (can't confirm this unfortunately, since I am stuck on Kepler).

Even when not using DMA-bufs, frame latency is reduced, when NVENC is available and the client does not disable AVC, since the content to upload won't be duplicated any more, before uploading the content (kills several milliseconds of latency).

See the commit messages for the individual changes.

Testing

To use gnome-remote-desktop with virtual monitors, switch the newly introduced gsettings key screen-share-mode to extend and connect with a remote desktop client, e.g. xfreerdp:

xfreerdp /v:<ip> /network:auto /u:<username> /p:<passwd> /size:1920x1080 /dynamic-resolution

/size defines the initial virtual monitor size
/dynamic-resolution allows resizing the virtual monitor during a session.

Note: Resizing the virtual monitor during the session currently triggers a crash in mutter. See mutter#1904 (closed) for more details (fixed by mutter!2270 (merged)).

Closes: #79 (closed)

Edited by Pascal Nowack

Merge request reports