Combine multiple search-file transactions in gs_plugin_packagekit_refine_async()
Note: This mostly appears to be an issue with PackageKit
. I'll move it to PackageKit
once we're sure that there is nothing related to GS in this issue.
Issue description:
Currently, we do a search-file
PackageKit
transaction for all installed desktop-application
packages whenever GNOME Software starts.
There are few issues with this:
-
It takes ~1 second per transaction ( 1 package / transaction ) for PK
apt
backend. So, in my case with around 70 installed GUI packages, all the 70 transactions takes 70 seconds in total ( 70 seconds just to find information on 70 installed GUI packages ) -
Since all 70 transactions are queued few seconds after GS starts, when we do any other action (
Click on "Develop" category button
,Click on "Updates" Tab
etc,) the new Clicked action will be added to the end of the PK queue, and the response to the action will only be available after ~70 seconds, which is quite bad. -
One core of the CPU shoots to 100% (
packagekitd
process) whenever GNOME Software is opened due to this issue. Though this is not a GS issue, it would be good to figure out why this happens, as we fix this issue.
APT delay: ( happens always )
I traced the delay to the AptJob::init()
( in PK ) which takes around 900ms.
PackageKit/backends/apt/apt-job.cpp:
bool AptJob::init(gchar **localDebs)
{
AptCacheFile *m_cache;
...
m_cache->Open(withLock);
...
}
PackageKit/backends/apt/apt-cache-file.cpp:
bool AptCacheFile::Open(bool withLock)
{
OpPackageKitProgress progress(m_job);
return pkgCacheFile::Open(&progress, withLock);
}
Here, pkgCacheFile::Open()
( which falls in apt
code base ), takes around 750ms.
This means 2 things:
- Currently, all
AptJob
s that are per-package will slow down GS considerably affecting overall user experience in Debian. - It also means that PK is using
AptCacheFile::Open()
in a wrong way or calling it in the wrong place, or maybe we should query for more info in a single PK query whenever possible (search-files pkg1 pkg2
rather thansearch-file pkg1
,search-file pkg2
etc ).
Below is the apt search-file
transactions with completion time ( some 4 contiguous entries )
search-file transaction /62629_ebdeedab from uid 1000 finished with success after 1102ms
search-file transaction /62630_edbaeabb from uid 1000 finished with success after 972ms
search-file transaction /62631_cdddaadd from uid 1000 finished with success after 1092ms
search-file transaction /62632_cdbdebed from uid 1000 finished with success after 1122ms
...
DNF delay: ( happens rarely )
I've not spent much time looking into this. But dnf
doesn't face this delay issue always. It happens in some corner cases, but not during every run.
Below is the normal dnf search-file
transactions with completion time ( first 4 entries )
search-file transaction /1584_cabcabac from uid 1000 finished with success after 433ms
search-file transaction /1585_cabcbddc from uid 1000 finished with success after 17ms
search-file transaction /1586_adbeeddb from uid 1000 finished with success after 21ms
search-file transaction /1587_cdabeabd from uid 1000 finished with success after 15ms
When we hit the corner case ( uncached case ? ) which causes the delay, there is one important difference compared to apt
delay mentioned above. There is no uniform per transaction delay in dnf
. The first search-file
transaction takes the most time. All other following transactions take very less time. So, I find the delay to be roughly same in this corner case, but with different transaction delay patterns ( perhaps due to the way dnf
/ apt
caching works ).
Below is the delayeddnf
search-file` transactions with completion time ( first 4 entries )
search-file transaction /1687_cdabeabd from uid 1000 finished with success after 54102ms
search-file transaction /1688_cdabcabd from uid 1000 finished with success after 12ms
search-file transaction /1689_cdaaaabd from uid 1000 finished with success after 23ms
search-file transaction /1690_cdabbabd from uid 1000 finished with success after 21ms
...
#1342 (closed) might be caused by this as well. Not sure though.