gimp_transform_polygon() transforms an (open or closed) polygon by
a GimpMatrix3, performing clipping to the near plane, to avoid
erroneously transforming points behind the camera.