C++ won't allow us to use GimpLayerMode in the API where we used to
have GimpLayerModeEffects.

Move GimpLayerModeEffects to libgimpbase/gimpcompatenums.h so it's
not in the API any longer, and instead typedef and define stuff in
libgimp/gimptypes.h, and adapt the compat enum registering code
accordingly.