GitLab

We want to specify file attributes up in the miner, so add a way to
let these further down to the TrackerCrawler.

So this is readily available to signal emission.

We are duplicating work, we have the TrackerCrawler inspecting every folder
and file, just to queue changed files, and then we go through these again
querying info individually.

This can be streamlined by pushing the GFileInfos from the TrackerCrawler
instead, so we don't need to query that info again. This commit just
does one step forward, letting this information available outside the
TrackerFileNotifier.

Instead of accumulating these files. Remove them from the TrackerFileNotifier
queues as soon as we know the file is unchanged, and at a time we have the
data/node handy.

When asking to crawl a directory, we always perform a info query on
the directory itself. However, most often this is a directory that
is being recursively inspected, thus comes from the TrackerCrawler
itself.

This brings 2 benefits:
- Most often we avoid a redundant file info query
- We now only call check_directory() once per directory, fixing
  the accounting of directories in our debug output.

These are stored as GObject data on the GFile, but detached early
on. Keep these tied to the file indefinitely, so they can be used
at later stages in indexing, they will be useful later on.

Delegate this (for the most part) on TrackerCrawler, as it is already
in the business of fetching GFileInfos for inspected files. This avoids
one extra stat() per folder.

We most often (except from monitor events) have a GFileInfo to pass on
to the TrackerIndexingTree. This will allow unifying some file queries
being done.

We check the parent folder (multiple times actually) before filtering
based on directory contents. Just avoid this here.

We already have a GFileInfo for the inspected file from the crawler,
push this forward so it can be used on file checks.

When crawling over a directory, we first check that the directory
exists. This is kinda pointless, as we can delay this error to
the moment we open the enumerator.

Instead of start/stop calls, and relying on signals. Make it an
async API, with a cancellable and finish() call.

Instead of maybe warning in place, ensure the errors get forwarded
further up to the TrackerCrawler.

Make file/directory checks through a func given through a setter, in order
to avoid signal emission overhead. This saves a bit of time as we emit up
to 2 signals per file.

The timer was unused in TrackerCrawler, except for setting it up
and destroying it.

This is no longer used, except for tests.

Directories in recursively indexed roots are doubly checked here,
once when processing the parent directory children, and another
when processing the directory and its children.

Avoid the latter, so we only go through these bits once.

This is going unused, replaced by the TrackerMinerFS LRU. We can remove
this in advance.

These are now unused, both querying and resetting.

This is supposed to happen with MONITOR | !CHECK_TIME. Luckily or
not, !CHECK_MTIME is currently broken (doubly! in handling, and in
setting it), anyhow this flag shouldn't mean "ignore newly found
files/directories", when fixed in future commits.

Drop this, we should really issue ::file-created from the regular
paths.

This is now down in the upper layers with the LRU.

We don't need to cache all of them for a cache to be useful, use
a LRU so we cache the most important/frequent at the time. This
also makes tracker_file_notifier_get_file_iri() entirely unused
and unneeded.

This is a generic LRU, that we'll use for our convenience.

This function is now unused.

If we trust the event flow, we shall receive a delete/move event for
that file in the future, so avoid handling it as a deletion here.

Besides the "peek into the future" layering break, we just used this
to see whether the file is a directory or not, we already have this
info from TrackerFileNotifier, so forward it.

At the TrackerFileNotifier level, we just check that the file is in
the TrackerFileSystem, even though it's not guaranteed that only eligible
files will get there. Later on, we have checks at the TrackerMinerFS
level to handle these move events we let through as if the destination file
were new. We can let TrackerFileNotifier issue ::file-created instead of
::file-moved then.

If we trust the event flow, we should not get move events for files that
we didn't now about earlier (either previously from the database, or recently
from events that were handled right before this one), so the checks at the
TrackerMinerFS level become moot.

These functions are unused now.

This was purely for tracker_miner_files_is_file_eligible() checks.
Since this function is gone, this can go away too.

Since commit 26f3ba3d, this is not as useful as it used to be. Since
the previous commit, it's completely unused, can be dropped now.

This is also done deeper on in the TrackerFileNotifier/TrackerIndexingTree,
we can avoid this specific check here.

This function is unused since commit 26f3ba3d.

This function is now unused.

The TrackerFileNotifier already knows this piece of info, so let it known
via the signals, instead of letting the upper parts figure it out by themselves.

These aren't generally hot paths, adapt thing so that we can eventually
drop TrackerFileSystem.

This is about the only place where we do actually rely on TrackerFileSystem
using "const" pointers to the same GFile above TrackerFileNotifier. It's
actually not that much of a hotspot, so use g_file_equal() here, and check
it early in the function to make it easier to read.

functional-tests: Validate SPARQL as part of extractor tests

Closes tracker#196

See merge request !277

Next stable release will be 3.1.0.

tracker-extract: Avoid the fluiddec gstreamer plugin

Closes #101

See merge request !279