This was caused by our caching of buffer lines (added in a15dbdfc)
because the BufferLines class can't tell when the filters it applies
change. The easy fix here is to manually clear the line cache on
comparison refresh, which is slightly over-zealous but probably fine in
most cases.

A better solution here would be to have the filters passed in to the
BufferLines class either be immutable or have a changed signal that
the class could connect to to do this cache invalidation on its own.