This is a huge quest to remove all caching from GtkIconview to simplify
the code. As it turns out, iconview performance is a joke, so the caches
are kinda unnecessary.

If we need caching, we can add it in a useful way later.