fontmap: be more defensive in patterns finalization

This clears a few pointers defensively in cleanup. It also uses a weak
pointer on the PangoFcFontMap so that we don't dereference the raw
pointer if we race during finalization.
5 jobs for wip/chergert/fontmap-cleanup in 7 minutes and 40 seconds (queued for 1 second)
latest
Status Job ID Name Coverage
  Build
passed #1175789
linux-fedora

00:02:00

passed #1175791
macos
macos

00:05:39

passed #1175790
win32-ps
msys2-mingw64

00:01:36

 
  Analysis
failed #1175792
asan
asan-build

00:01:56

 
  Docs
passed #1175793
reference

00:01:38

 
Name Stage Failure
failed
asan-build Analysis
/builds/GNOME/pango/_build/report.xml: found 1 matching files and directories 
/builds/GNOME/pango/_build/hello.png: found 1 matching files and directories

/builds/GNOME/pango/_build/fontlist.txt: found 1 matching files and directories

Uploading artifacts as "archive" to coordinator... ok
id=1175792 responseStatus=201 Created token=JZzzGW8D
Uploading artifacts...
/builds/GNOME/pango/_build/report.xml: found 1 matching files and directories

Uploading artifacts as "junit" to coordinator... ok
id=1175792 responseStatus=201 Created token=JZzzGW8D
Cleaning up file based variables
ERROR: Job failed: exit code 1