Refactor threading for refine job
This branch refactors the threading for the refine job, splitting it out into a new GsPluginJobRefine
subclass of GsPluginJob
, and changing the vfunc in all the plugins from gs_plugin_refine()
to an asynchronous refine_{async,finish}()
on GsPluginClass
.
This branch basically sets the architecture for the rest of the threading rework.
- Different
GsPluginAction
s will be moved to subclasses ofGsPluginJob
, so that more job-specific arguments, options and methods can be exposed. - The plugin functions which are currently looked up using
dlsym()
will all slowly be moved over to be vfuncs onGsPluginClass
. - Plugins will implement their own threading (if they want); some plugins may not need threading. Those which do need threading will likely use a single worker thread for all their operations, rather than an unbounded number of worker threads from a shared thread pool. There’s a fairly even split between plugins which implement
refine_async()
using a thread, and those which implement it using one or two main context callbacks. - The PackageKit plugin refine code had to be significantly reworked to make it asynchronous. A benefit of that is that it now does a lot of its D-Bus calls in parallel, rather than in series. This should speed things up, but I haven’t profiled it or spent any time on performance. It’s possible there are some silly performance bugs which will need addressing in future.
The performance of this branch is subjectively about the same as before. I haven’t measured anything. Everything seems to work, but there may be regressions; unless I’ve missed anything large I’d prefer to handle them in follow-ups to avoid this MR getting bogged down in review for a long time and bitrotting.
Sorry this MR is so large. I couldn’t find a way to split it up further while keeping gnome-software working correctly. The branch may not compile between some commits, and almost certainly doesn’t work correctly, due to so many things being refactored at once.
A lot of the commits are quite similar (port plugin X to use asynchronous refine) and should be fairly easy to review. The PackageKit changes apply in series: there’s a set of preparatory commits which refactor the code while keeping it functionally identical to before; then the functional changes are in a subsequent commit.
Future MRs for threading rework won’t be this complex. The refine action is a really complicated one, implemented by almost all the plugins, and with some odd internal uses which complicate it further (other plugin jobs can set their refine-flags
property and that triggers an internal refine operation).
Follow-up work which needs to be done shortly after this MR:
- Make
run_refine()
properly async (comments) - Merge
packagekit-refresh
andpackagekit-refine-repos
plugins intopackagekit
(as per !1131 (closed)): !1141 (merged) -
Refactor download code in
fedora-pkgdb-collections
Helps: #1472