We can set individual "filters" for the different pipelines. Use that to replace the software format conversion and flip for the viewfinder with the corresponding GL elements.
Unfortunately this needs a fix in Gstreamer. Once that has landed it will allow us more flexibility to try various things such as using GL elements for the capture cases as well.
Draft as it depends on https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5527