Skip to content

Refactor threading for refine job

Philip Withnall requested to merge pwithnall/gnome-software:refactor-threads5 into main

This branch refactors the threading for the refine job, splitting it out into a new GsPluginJobRefine subclass of GsPluginJob, and changing the vfunc in all the plugins from gs_plugin_refine() to an asynchronous refine_{async,finish}() on GsPluginClass.

This branch basically sets the architecture for the rest of the threading rework.

  • Different GsPluginActions will be moved to subclasses of GsPluginJob, so that more job-specific arguments, options and methods can be exposed.
  • The plugin functions which are currently looked up using dlsym() will all slowly be moved over to be vfuncs on GsPluginClass.
  • Plugins will implement their own threading (if they want); some plugins may not need threading. Those which do need threading will likely use a single worker thread for all their operations, rather than an unbounded number of worker threads from a shared thread pool. There’s a fairly even split between plugins which implement refine_async() using a thread, and those which implement it using one or two main context callbacks.
  • The PackageKit plugin refine code had to be significantly reworked to make it asynchronous. A benefit of that is that it now does a lot of its D-Bus calls in parallel, rather than in series. This should speed things up, but I haven’t profiled it or spent any time on performance. It’s possible there are some silly performance bugs which will need addressing in future.

The performance of this branch is subjectively about the same as before. I haven’t measured anything. Everything seems to work, but there may be regressions; unless I’ve missed anything large I’d prefer to handle them in follow-ups to avoid this MR getting bogged down in review for a long time and bitrotting.

Sorry this MR is so large. I couldn’t find a way to split it up further while keeping gnome-software working correctly. The branch may not compile between some commits, and almost certainly doesn’t work correctly, due to so many things being refactored at once.

A lot of the commits are quite similar (port plugin X to use asynchronous refine) and should be fairly easy to review. The PackageKit changes apply in series: there’s a set of preparatory commits which refactor the code while keeping it functionally identical to before; then the functional changes are in a subsequent commit.

Future MRs for threading rework won’t be this complex. The refine action is a really complicated one, implemented by almost all the plugins, and with some odd internal uses which complicate it further (other plugin jobs can set their refine-flags property and that triggers an internal refine operation).

Follow-up work which needs to be done shortly after this MR:

Helps: #1472

Edited by Philip Withnall

Merge request reports