VTE based terminal is not handling cursor movement outside scroll region properly
So the initial problem I had is nested SSH Tmux displaying my cursor above the ZSH prompt. So I opened an issue in the Tmux issue tracker here: https://github.com/tmux/tmux/issues/1952 At the end, the maintainer of Tmux found that this issue caused by VTE based terminal (Gnome-terminal and Termite that I'm using confirmed). I tried other non VTE based terminal and the problem is gone. This is the way to test it.
printf '\033[r\033[H\033[2J' printf '\033[4;7r\033[3;1Haaa\033[Axxx' sleep 10
Gnome-terminal will output:
While non VTE based terminal will output: