Don't invalidate parent if it didn't change

This looks like a leftover excess invalidation from when the surrounding code was refactored to not just be called on parent changes but also when repositioning inside the same parent in commit 507016ca

Ivan Molodetskikh found this problem in #3334 (comment 1445873) which contains a longer analysis of this problem and the performance reductions it causes.

Related: #3334 (closed)