This mode could also shrink the selection, plus the starting point would
seem somewhat arbitrary (actually dependent on the dragging direction of
the last selection).

Made this mode more consistent by only allowing it to extend the selection,
only in one direction for each operation, and so it keeps the current
selection as a minimum.