fontmap: be more defensive in patterns finalization
This clears a few pointers defensively in cleanup. It also uses a weak pointer on the PangoFcFontMap so that we don't dereference the raw pointer if we race during finalization.
I haven't tested to see if this actually fixes things since it's hard for me to reproduce, but related to #537 (closed)