Skip to content

tracker-miner-fs: Do not use unrestricted queries moving files

Carlos Garnacho requested to merge wip/carlosg/move-perf into master

When a directory is renamed and a recursive renaming of the nfo:FileDataObjects starts, we use very widely generic queries to mirror all the data from the old nfo:FileDataObjects to the new ones with the updated URIs.

This { ?s ?p ?o } query results in the triples virtual table being queried in the most generic way possible, thus all values are returned. This makes the updates that handle these recursive moves dependent on the number of elements in the database.

The use of unrestricted queries is actually discouraged in Tracker documentation [1], so avoid falling in our own foretold mistakes. Since we do know the data we expect to exist in a nfo:FileDataObject in both the tracker:FileSystem and content graphs, make the relevant queries completely specific about the properties being moved from the old nfo:FileDataObject to the new one.

The performance improvement when handling move events goes from barely noticeable to massive, depending on the database size. For ~130K elements indexed, moving a directory containing 180 files went down from 48-50s to ~2s.

[1] https://gnome.pages.gitlab.gnome.org/tracker/docs/developer/performance.html#avoid-queries-with-unrestricted-predicates

Closes: #228 (closed)

Merge request reports