Skip to content

tracker-miner-fs: Preserve URN of nfo:Folders across updates

Carlos Garnacho requested to merge wip/carlosg/fix-flaky-test into master

Commit ad34c7d6 tried to handle updates on folders like we do with any other nie:InformationElement (i.e. deleting it and waiting for a fresh one), with the added trickiness that child files must point to the nfo:Folder, old and new.

This worked for the most part, but still there was a race condition it could not handle, and triggered randomly by the corresponding test (commit 8d1de376), it goes like:

  • child file is updated
  • update is requested on the parent (indexed) folder
  • both things go in that order up to TrackerMiner::process_file(), the order is right and neither task blocks, both get processed asynchronously "simultaneously"
  • In the async g_file_query_info() processing, both tasks flip order.
  • The task for the parent folder completes first, it changes the nfo:belongsToContainer relation of all children, but the new file is still not inserted. The SPARQL is queued up.
  • The task for the child file completes second, it relates to the parent folder, but when looking up its URN the old one is found (as we didn't process the SPARQL for the parent folder yet).

This order happening naturally in the event queue would result in the queue handling "blocked" until the parent folder is processed, here we go with it, and end up triggering random test failures like:

Traceback (most recent call last): File "/builds/GNOME/tracker-miners/tests/functional-tests/miner-basic.py", line 378, in test_10_folder_update self.assertEqual(self.__get_parent_urn(document), AssertionError: 'urn:bnode:87544fe5-06e6-487e-93a7-cc8f72e544af' != 'urn:bnode:a9c70d9c-c365-4fe6-8048-889eb6d9ca9e'

  • urn:bnode:87544fe5-06e6-487e-93a7-cc8f72e544af
  • urn:bnode:a9c70d9c-c365-4fe6-8048-889eb6d9ca9e

Actually, dropping the nie:InformationElement is not that crucial with nfo:Folders (since we know they'll become a nfo:Folder again), preserving those avoids the need to update nfo:belongsToContainer in children altogether, and makes the race condition moot (as the URN is preserved across the update).

Merge request reports