As has been discussed above, this is unfixable in X11 due to how it renders frames (renders all displays in a given screen at the same time, or vice-versa, or something like that).
TBF I'd switch over to Wayland unless you have a solid reason to remain with X11 ...