During the initial crawl, newly created files and directories may be ignored
The tracker-miner-fs does an initial crawl on startup of the configured indexing locations. During this time, monitors are set up for each directory so that changes on disk are detected. There is a period between a the contents of a directory being listed and the monitor being added, and any changes to the filesystem that take place in this window are ignored.
This caused intermittent test failures in the functional-tests (#56 (closed) #57 (closed) #62 (closed)). The minertest fixture created its test data after starting the miner, leading to occasional cases where files were not indexed because of this race condition.
Crawling starts in tracker-crawler.c with
data_provider_begin_cb() callback enumerates directory contents with
g_file_enumerator_next_async(). Once this completes,
::check-directory-contents signal. The
crawler_check_directory_contents_cb() method in tracker-file-notifier.c will call
tracker_monitor_add() for the directory, at which point we will start to receive change notifications for the directory.
To work around this problem, you can call
tracker_miner_manager_index_file() to trigger re-indexing of any important content.
To fix this problem, we would need probably need to record the mtime of each directory we find, and if the mtime changes before we finish emitting
::check-directory-contents then we need to re-crawl it. We would need to thoroughly test the change to make sure this doesn't have a big performance impact.