Skip to content

DRM vs. Cogl pixel format confusion in Wayland implementation

https://gitlab.gnome.org/GNOME/mutter/blob/master/src/wayland/meta-wayland-dma-buf.c#L88

It seems that DRM_FORMAT_XRGB8888 is taken equal to COGL_PIXEL_FORMAT_RGB_888, but when I was reading the definition of CoglPixelFormat I don't think that's right. As far as I understand, COGL_PIXEL_FORMAT_RGB_888 is 3 bytes per pixel, not 4; as in, depth=24 bpp=24 in X11 parlance.

But I also cannot figure out what DRM_FORMAT_XRGB8888 should actually map to. CoglPixelFormat definition seems to imply that it maps to COGL_PIXEL_FORMAT_BGRA_8888 but you also need an additional flag in all users to say that the alpha channel must be ignored.

cogl-types.h says:

  • Note: Cogl avoids exposing any padded XRGB or RGBX formats and
  • instead we leave it up to applications to decided whether they
  • consider the A component as padding or valid data. We shouldn't
  • change this policy without good reasoning.

Therefore one cannot simply convert from the DRM format to a Cogl format, can you?

When I said BGRA in the above, it's not a typo. cogl-types.h says this:

  • Pixel formats used by Cogl. For the formats with a byte per
  • component, the order of the components specify the order in
  • increasing memory addresses. So for example
  • %COGL_PIXEL_FORMAT_RGB_888 would have the red component in the
  • lowest address, green in the next address and blue after that
  • regardless of the endianness of the system.

So while DRM formats are given as bits of u32 on little-endian, Cogl formats use the reverse order in naming formats. That means in the above mentioned file, also this is wrong:

    case DRM_FORMAT_ARGB8888:
      cogl_format = COGL_PIXEL_FORMAT_ARGB_8888_PRE;

If the docs are to be trusted, the Cogl format should be COGL_PIXEL_FORMAT_BGRA_8888_PRE instead.

However, I assume this code produces the correct colors, at least with DRM_FORMAT_ARGB8888, because this path is used by practically all GL apps, right?

The reason I looked into this is that I need to convert between DRM and Cogl pixel formats, and I was looking for examples on how to do that. But documentation and the example I found seem to disagree.

Or, maybe the Cogl format inferred in meta_wayland_dma_buf_buffer_attach () is never used... let's see. cogl_egl_texture_2d_new_from_image () has this comment:

/* NB: The reason we require the width, height and format to be passed

  • even though they may seem redundant is because GLES 1/2 don't
  • provide a way to query these properties. */

For me, Mutter runs on OpenGL 4.3 Core profile, not GL ES. I guess the format is never actually used then, but if one tried to run Mutter with GL ES 2, you'd be in trouble.

Supposedly I can therefore summarize this bug report as:

meta_wayland_dma_buf_buffer_attach () converts DRM pixel format to Cogl pixel formats incorrectly, but it usually has no visible effects because the format is not used.