Configuring displays in regards to color depth, and bandwidth constraints
Feature summary
Right now the settings that gnome-shell exposes for controlling various display related settings are really barebone. This doesn't currently matter nearly as much as it will once HDR is more well supported in mutter and gnome-shell.
The DRM core in the kernel kernel by default attempts to use the highest BPC possible on a display, so long as the display can support it. This support depends on a couple of different factors: what the GPU can support, what the monitor can support, and the amount of bandwidth available on the relevant display connector. Currently, userspace can force a lower bpc through the max_bpc connector property.
Currently, we don't really support max_bpc very well in the kernel for DisplayPort MST connectors in any of the drivers. I've been planning on doing this for a while, and as of late I've finally gotten the time to start seriously working on this again. This is also the main reason I'm filing this issue, because things are going to get a bit more complicated once we start supporting this with MST.
See, since DisplayPort MST is used for daisy-chaining multiple displays on a single DisplayPort connector, all of these connectors share bandwidth with each other. This can lead to some rather fun situations: Let's say we have an MST hub, on one port is a fancy super cool 8k display, and on the other two ports there's some lame old 1920x1080 displays. Now, if the user enables the 8K display on its own and there's enough bandwidth to support it, things will run fine. But if we start enabling the other displays at the same time, it's entirely possible we might start running out of available bandwidth and not have enough to enable additional displays. There's other possibilities as well: if the kernel notices that we can enable DisplayPort stream compression on certain displays, or perhaps use a lower BPC or lower refresh rate, it may do that automatically in order to make the mode configuration of each display fit in the available bandwidth.
And then there's even the possibility that the hub might suddenly decide it can't support it's current link speed and downgrade itself - resulting in less available bandwidth for all displays involved, which might potentially require userspace to choose different display modes in order to make things fit (fwiw, this is the link_status property, which we also don't have fully implemented for MST yet). Note though that from the kernel's perspective we do everything in our power to make sure this last happens as little as possible, and I've only ever actually ran into it on one dock in my entire DisplayPort hardware repertoire.
So with the context given, this feature is mainly about teaching mutter how to use the max_bpc setting with mst, how to handle link_status changes with MST.
How would you like it to work
Imagine we have a photographer using gnome-shell, who needs to ensure that the images they're working on have as accurate colors as possible on their fancy 8K display. This of course implies having HDR support. Now, similar to the example I gave in the previous section such a user may have this display on a laptop dock which also has multiple other displays hooked up to it. If this photographer uses their 8K display as the main display for working on images, they're going to want to be able to tell gnome "Hey-do whatever it takes to make this single 8K display operate in HDR mode, even if that means downgrading the image quality on other displays. Also, tell me if that's literally impossible".
So, for allowing the user to control on specific displays to enable for explicitly requesting deep color support on specific displays, this would likely be as simple as introducing a "color depth" setting in gnome's control center, and then mutter could use atomic modesetting to figure out if the user's requested settings are possible or not. Currently there's only one issue I see with this: max_bpc only allows you to limit the maximum BPC that the kernel will use for a particular display. I'd imagine in most cases a user would want the opposite, where they want to ensure a particular display is in HDR mode. If this really is a problem, I think I could add a min_bpc property on the kernel side to make this easier to implement in gnome-shell and other compositors.
The next part, after I've implemented in the kernel at least, would be handling the link_status transitions on MST. This part is a bit more complicated, for instance let's say we have a user whose display configuration looks like this:
Laptop → Dock → Monitor
→ DisplayPort MST hub → Monitor
→ Monitor
The simplest link_status transition to handle would be if the connection between the laptop and the dock was suddenly downgraded to a lower link speed. In this case, link_status would be set to bad on all three monitors since they all may need their display configuration changed. Once userspace commits a new mode on any of the monitors which opens up enough bandwidth for the rest of them to operate, link_status would go back to good on each monitor.
It's also possible however (at least I think so according to the DP spec, but haven't run into this in the real world yet) that the connection between the dock and the external DisplayPort MST hub could have it's link rate downgraded, in which case the two monitors connected to that MST hub would have their link_status set to bad, while the rest of the topology would be fine. In that case, the display configuration on either of the two monitors at the end of the topology would need to be reset or changed by userspace.
Relevant links, screenshots, screencasts etc.
None yet beyond what I've linked here but feel free to ask questions. FWIW, I'm also likely going to get the sway/kde folks in this discussion so they're kept in the loop.