Wayland: Add support for YUV formats
Motivation
Supporting YUV formats allows applications to offload YUV->RGBA conversions to the compositor. This alone is not a big win, however the compositor in turn may be able to offload the conversion to the display hardware. This can improve performance and reduce energy consumption substantially, especially on mobile or embedded devices.
Supporting this feature also makes Mutter a more attractive development platform for people working on software for e.g. embedded devices - this is partly my personal motivation.
Status
As a start we'll support only the following formats:
- single-plane packed
YUYV - two-plane
NV12 - triple-plane
YUV420.
Shaders were tested on
- modern Intel (Iris/Skylake/GL3)
- old Intel (Crocus/GMA4500/GL2)
- modern AMD (RadeonSI/Radeon 6700XT/GL3)
- common ARM (Panfrost/RK3399/GL3)
Hardware offloading was successfully tested with fullscreen clients for YUYV and NV12 on Intel and AMD.
How to test
Depending on the hardware, the following commands are helpful for testing:
-
weston-simple-dmabuf-v4l [-f NV12]- format support depends on the camera
-
mpv --hwdec=vaapi --vo=dmabuf-wayland sintel_trailer-1080p.mp4 --fs --vf=format=[yuyv422|nv12|yuv420p]-
mpvfrom git / > 35.1 required to avoid freezes. - format conversion via
--vf=format=may only work on Intel.
-
-
gst-launch-1.0 filesrc location=sintel_trailer-1080p.mp4 ! qtdemux ! h264parse ! vah264dec ! vapostproc ! waylandsink fullscreen=true- likely broken on Intel until explicit modifier negotiation is implemented in the va and waylandsink plugins. Confirmed to work when forcing matching modifiers or using upcoming explicit modifier support.
On Intel you may also test some WIP test clients.
This MR is a rebase of https://gitlab.gnome.org/GNOME/mutter/-/tree/wip/nielsdg/meta-multi-texture-wip by @nielsdg. The incomplete support for shm-buffers and EGLimage have been removed again, only DMABUF is supported for now.
Closes #1366 (closed)