Skip to content

Add new Memoryformats and CICP matrix coefficients

This branch does a lot of interwoven work that depends on each other so I couldn't split it apart into individual MRs without breaking stuff. So it's all in here now.

highlevel features

  • Make GdkMemoryFormat API be block-based and allow multiple planes. Memory formats are now defined by blocks of size W x H pixels with X bytes each. Textures of a format must have sizes that are multiples of the block size. For all existing formats, the blocksize is 1x1.

  • Encapsulate all the data for memory in a GdkMemoryLayout and update all internal APIs to use it. In particular, this API now allows copying textures unmodified to/from Vulkan.

  • Add a bunch of single- and multiplane GdkMemoryFormats with different block sizes, so that all previously supported dmabuf formats now have a memoryformat.

  • Consolidate all the mapping tables between Vulkan formats and GL formats and dmabuf formats in one place: gdkmemoryformat.c. This allows quickly mapping formats from one API we use to any other. And it makes it reasonably easy to add new formats, because they "just" need to be added to the big table.

  • Using this new table, simplify dmabuf downloads by not having them do any form of colorspace conversion. In fact, treat YUV dmabufs such as NV12 literally like G8_B8R8_420 and let colorspace conversions be done by the colorstate code.

  • Add matrix conversion and range expansion to the CICP colorstates, so they can actually do that job.

  • Add a GskGpuConversion value to the GPU renderer, so it can track the conversion that happens to textures inside renderes.

  • Use these conversions to accurately manage matrix coefficients and range conversion. This is tricky in particular because Vulkan has access to the raw YUV data of textures (using aspect planes) while GL can only access the data via the framebuffer (aka the color aspect).

  • While at it, reinvent how textures can be specified in node files, so I can actually debug this stuff. That includes pulling udmabuf support into it, so node files can now specify dmabuf textures, complete with multiple planes and the fourcc of choice.

API additions

  • 23 new YUV memoryformats, including "P010", "NV12" and "YUV420".

  • GdkCicpParams now supports various ranges and matrix coefficients.

  • gdk_memory_texture_builder_set_stride_for_plane() and gdk_memory_texture_builder_set_offset() to create multi-plane memory textures.

  • gdk_texture_downloader_download_bytes_with_planes() to download multi-plane textures.

The GStreamer sink has been updated to use all these things - it sets colorimetry correctly and supports YUV memory textures in those 23 new features.

gtk4-image-tool convert can now write out texture node files in every supported memory format.

Edited by Benjamin Otte

Merge request reports

Loading