Timeline clips filmstrip thumbnailer does too much background work and slows down / blocks the rest of the UI
@jeff
Submitted by Jeff F.T. Assigned to Jeff F.T. @jeff
Description
To reproduce in master:
-
Have "normal" computer hardware. Instead of a Core i7 with a solid-state drive, use a Core 2 Duo with a regular hard disk drive. However, the issue will still be noticeable even on the super-fast computer, just a little bit less so. We can't expect users to be running brand new computers bought last month, we should expect them to run machines that are five years old.
-
Have a project file with a bunch of clips in the timeline. Texture (from http://jeff.ecchi.ca/public/sample-pitivi-projects/ ) can be a good test case.
-
In ~/.cache/pitivi, rename or delete the "thumbs" folder.
-
Load the project.
Result:
- Project loading takes a LOT longer than before. It used to be nearly instantaneous before the implementation of the clutter timeline and new thumbnailer.
- Extremely sluggish/non-responsive UI during (and after) project loading. You will feel it, guaranteed.
On a Thinkpad X61, the system load goes to 14 when trying to load the “Texture” sample project.
Yet, as far as we have been told, we are not blocking on I/O anymore (in theory), so maybe we are now starving the CPU by doing too much work at once and at the wrong time, preventing the UI from having enough ressources to be reactive. Would be nice to do profiling to be sure.
The current thumbnailer implementation does two things:
- It generates thumbnails for the visible portion of clips in the timeline, in higher priority
- It does background processing to process the parts that are not currently visible
Potential avenues to investigate or solve the problem:
-
Defer work to when nothing else is happening. Thumbnailing should never happen while we're loading a project, playing or rendering. Be able to pause and resume background thumbnailing.
-
Do less concurrent work. Throttle the background thumbnailing with a scheduler? Depending on the number of available CPU threads/cores. Maybe always try to leave one core free, especially if the pitivi window is not the focus (ie the user is doing something else)? Or, more simply, instead of processing all clips at the same time, perhaps limit to processing only one clip at a time for the background queue?
-
Previously, it committed (writes the clip's thumbnail cache sqlite file to disk) for every single thumbnail in the clip, which quickly destroyed performance by saturating with unnecessary I/O. The current implementation now commits only once the clip has been entirely processed, which is lighter in terms of I/O but also a bit overzealous, in the sense that it would be sufficient to commit using a timer (ex: every 30 seconds or when the clip processing completes, whichever comes first). Otherwise, if you try to thumbnail a clip that is two hours long, nothing will ever get written to disk until it has been thumbnailed entirely (which can take hours)
Somewhat related (or perhaps to be obsoleted): https://bugzilla.gnome.org/show_bug.cgi?id=T2326#c21
Imported from https://bugzilla.gnome.org/show_bug.cgi?id=700610