Skip to content

Speed track

Carlos Garnacho requested to merge wip/carlosg/speed-track into master

Some rather substantial refactors to improve performance of tracker-miner-fs performance, both first-time and "regular" startup.

  • TrackerFileSystem is dropped away. Its aggressive caching brings some overhead that doesn't justify the benefits. It is replaced by plain g_file_equal() checks, and a LRU in TrackerMinerFiles to cache a small number of URNs.
  • TrackerCrawler has been sanitized, made async API, and removed of all its signals. All of this aiming to reduce the overhead involved in checking every file and directory during startup.
  • TrackerCrawler is made more central. We use it to query GFileInfos for every file and folder, just to throw them away after queuing the filesystem changes found. Later on all those changed files are inspected again and those GFileInfos indexed. After these changes, the GFileInfos produced by the TrackerCrawler are those seen by the TrackerMinerFiles, this API does not need to be async anymore, which allows processing large batches of files without that much idle overhead.
  • In general, less stat() calls happen all over the place, should be one per file now.

I'll post better numbers, but these changes seem promising, first time indexing (10535 folders, 5703 files) went from ~45s down to ~18s, and second-time startup (no changes) from ~5.8s to ~2.2s (as reported by G_MESSAGES_DEBUG=Tracker)

Edited by Carlos Garnacho

Merge request reports