Revert "file chooser: Allow activating without double-click"

This reverts commit fb0a13b7.

It's already reverted in master via
c8a6a113, so let's not leave subtle
behavior changes that would make a gtk3->gtk4 migration. And just like
the commit message of the revert already mentions: it didn't really make
anybody happy anyway.