I'm not sure if this is a Games bug, or a GTK+ bug, or something with Mutter, or with graphics driver, but it happens.
My guess is that we're doing something wrong in RetroGLDisplay, such as not freeing GL context.
RetroGLDisplay