Pipewire streams include window decorations and use decoration sizes for clamping
Note: although this is likely related to #2099 (closed), the particular issues I'm running into feel sufficiently different enough to create a new issue.
Affected version
- OS: Arch Linux
- kernel version: 6.1.31 (hardened)
-
mutter
version: 44.2 -
gnome-session
version: 44.0
Bug summary
I'm trying to use PipeWire to capture a window via OBS. This affects any program which uses PipeWire for screen capture.
There are two issues with the way window captures work:
- The decorations of the window, including the title bar and a shadow around it, are included in the stream with no option to be omitted.
- Although streams can capture windows even if they're not on-screen (either partially past the screen border or on a separate workspace), streams are clamped to the screen resolution, meaning that the combined decorations + window resolution is artificially clamped to the screen resolution.
This manifests into a third issue: if a window's drawn region is set to be the screen resolution, even if the window can be moved so that its drawn region occupies the whole screen, the stream will instead draw the portion of the window that would be viewed if the edge of the shadow lined up to the top-left of the screen, cutting off the bottom-right portion of the window viewport.
Steps to reproduce
Problem 1:
- Capture any window using a program like OBS.
- Observe that the shadow and top bar are included in the stream, with no way to remove them.
Problem 2:
- Open a window with a fixed resolution such that its drawn area exceeds either the horizontal or vertical resolution of the screen.
- Capture this window using a program such as OBS.
- Observe that the clamping behaviour occurs.
What happened
There are two primary problems with the way GNOME captures windows. First, the decorations for windows (the window title bar and the shadow around it) are included in the stream with no option to remove them. While this can create nice screenshots when capturing small windows, it causes issues for capturing games and other windows. If an extension like the "undecorate windows" extension are used to manually remove the decorations from these windows, then the decorations vanish from the screen, although these changes are not permanent and can be reset upon changes in window focus, which are common when adjusting streaming setups. (Common scenario: focus streaming program to adjust stream. Refocusing the thing you're streaming causes new adjustments to be required. Repeat.)
If the above issue were the only problem, albeit tedious, the decorations could be manually cropped out of the stream. This is problematic because the size of the shadow and top bar are not guaranteed to never change (why would they be?) and thus updates to GNOME could require adjusting any existing setup. This is also a terrible user experience. (Source: I am a user. I hate it.)
However, because of the clamping behaviour described, decorations cannot be left in if the combined window + decoration size exceeds the window resolution. This feels like an arbitrary restriction that was added, and should be fixed. Note that because shadow is included even if the title bar + drawn region don't exceed the screen resolution, the stream will.
What did you expect to happen
There should be an option to omit window decorations from PipeWire streams. If the protocol cannot possibly accomodate this, then they should never be included in PipeWire streams, and any program which wishes to include these decorations in streams (like screenshotting tools) should be required to use a separate protocol or protocol extension, potentially a GNOME-specific one.
Because streams are designed to allow windows that are partially offscreen, window capture specifically should not be limited to the resolution of the screen, and instead should be sized based upon the resolution of the window. If decorations are included in the stream, that should include the decorations.
Since streams can already customise whether the cursor is included, I think that customising whether decorations are included isn't a particularly outlandish request. Note that windows which include semantic information inside their title bar (for example, Firefox merges the tab bar with the close button) will still count the title bar as part of the window, not its decorations, and these requests would still preserve streaming that semantic information.
Relevant logs, screenshots, screencasts etc.
Here is a screenshot of a window I'm trying to capture using the built-in snapshotting tool. Notice that decorations are added, but nothing is cropped out:
Notice that because my screen is 5120x1440p, the stream is not cropped horizontally, only cropped vertically.