Instead, add a gimp_layer_mode_get_format() function, which takes
the layer mode, composite space, and blend space, and returns the
I/O format.

Currently, we always use the composite space format as the I/O
format.  This simplifies gimp_composite_blend(), and gives us
composite-space support for the "special" layer mode ops for free.