macos: restore unfullscreen frame with style mask
This doesn't give us appropriate results if we use the window delegate. Instead, we need to adjust the frame at the same time we change the style mask so that we end up in the same location.
This also makes the transition work, albeit we don't have a lovely way to integrate there today and that might take some magic to improve in the long term if someone is so inclined.