common: Don't Handle list-row switch state manually

This was a hack added for libhandy-0, which is no longer required. Also, this hack results in the switch not working when gtk animations are turned off.