Here's some rewrite of handling layouts. The only functional change is that multiple graphical buttons are no longer allowed to control the same interaction. That feature wasn't used anywhere to my knowledge, so no one will miss it.
The biggest benefit from the cleanups is that cacheable information about layouts has been separated from runtime state, making it a bit easier to understand the whole thing. It also opens the door to layout caching, which can be helpful here: #337
Finally, this moves all the mutability/side effects closer together (state updates are less tied to logic, but input method events are still interleaved), moving us closer to getting all logic separate from all updates. That's a unit testing improvement, and also brings panel contents handling nearer to the state machine/actor model known from state.rs.