Don't rely on the exact modifier being pressed or released. Instead,
check if only the right modifier is pressed after *each* modifier
change, and switch to color picking if it is; disable color picking
otherwise. This greatly reduces the risk of missing the user's wish to
pick colors because of other modifiers being pressed and released in
whatever order.

Probably fixes bug #734743.