Unfullscreen action allowed in unexpected modes

Currently the unfullscreen action (swiping down from the top of the monitor) is enabled for Shell.ActionMode.ALL & ~Shell.ActionMode.LOCK_SCREEN, which means it includes UNLOCK_SCREEN and OVERVIEW. The former is quite clearly an oversight and the latter does not seem to make much sense either. In any of the modes it is currently enabled it will also prevent clicks in the top 20 px of the panel while a fullscreen window is focused.

The question is whether it should be limited to NORMAL or if some of the additional modes make sense. Also if it was limited to NORMAL would it even be possible to trigger it? Currently the events don't seem to reach the action, but I guess that might change with the upcoming grab rework. But then it might also be too easily triggered accidentally in games or so.