GitLab

Make file/directory checks through a func given through a setter, in order
to avoid signal emission overhead. This saves a bit of time as we emit up
to 2 signals per file.

The timer was unused in TrackerCrawler, except for setting it up
and destroying it.

This is no longer used, except for tests.

Directories in recursively indexed roots are doubly checked here,
once when processing the parent directory children, and another
when processing the directory and its children.

Avoid the latter, so we only go through these bits once.

This is going unused, replaced by the TrackerMinerFS LRU. We can remove
this in advance.

These are now unused, both querying and resetting.

This is supposed to happen with MONITOR | !CHECK_TIME. Luckily or
not, !CHECK_MTIME is currently broken (doubly! in handling, and in
setting it), anyhow this flag shouldn't mean "ignore newly found
files/directories", when fixed in future commits.

Drop this, we should really issue ::file-created from the regular
paths.

This is now down in the upper layers with the LRU.

We don't need to cache all of them for a cache to be useful, use
a LRU so we cache the most important/frequent at the time. This
also makes tracker_file_notifier_get_file_iri() entirely unused
and unneeded.

This is a generic LRU, that we'll use for our convenience.

This function is now unused.

If we trust the event flow, we shall receive a delete/move event for
that file in the future, so avoid handling it as a deletion here.

Besides the "peek into the future" layering break, we just used this
to see whether the file is a directory or not, we already have this
info from TrackerFileNotifier, so forward it.

At the TrackerFileNotifier level, we just check that the file is in
the TrackerFileSystem, even though it's not guaranteed that only eligible
files will get there. Later on, we have checks at the TrackerMinerFS
level to handle these move events we let through as if the destination file
were new. We can let TrackerFileNotifier issue ::file-created instead of
::file-moved then.

If we trust the event flow, we should not get move events for files that
we didn't now about earlier (either previously from the database, or recently
from events that were handled right before this one), so the checks at the
TrackerMinerFS level become moot.

These functions are unused now.

This was purely for tracker_miner_files_is_file_eligible() checks.
Since this function is gone, this can go away too.

Since commit 26f3ba3d, this is not as useful as it used to be. Since
the previous commit, it's completely unused, can be dropped now.

This is also done deeper on in the TrackerFileNotifier/TrackerIndexingTree,
we can avoid this specific check here.

This function is unused since commit 26f3ba3d.

This function is now unused.

The TrackerFileNotifier already knows this piece of info, so let it known
via the signals, instead of letting the upper parts figure it out by themselves.

These aren't generally hot paths, adapt thing so that we can eventually
drop TrackerFileSystem.

This is about the only place where we do actually rely on TrackerFileSystem
using "const" pointers to the same GFile above TrackerFileNotifier. It's
actually not that much of a hotspot, so use g_file_equal() here, and check
it early in the function to make it easier to read.

functional-tests: Validate SPARQL as part of extractor tests

Closes tracker#196

See merge request !277

Next stable release will be 3.1.0.

tracker-extract: Avoid the fluiddec gstreamer plugin

Closes #101

See merge request !279

libtracker-miner: Add missing ref to SPARQL buffer task

Closes #130

See merge request !278

This is a module that attempts to decode MIDI and send it to the
soundcard. We don't want such access triggered from tracker-extract
(and it's actually caught by our seccomp sandbox). Avoid this plugin
altogether.

Fixes: #101

On error paths, we let tracker_sparql_buffer_push_finish() still return
the underlying TrackerTask, together with the error, so we can detach
it properly.

However, in this situation we are not adding an extra task ref, while
the callback expects to own a reference. This causes refcounting issues
and eventually invalid reads.

Fixes: #130

The existing tests served to check that correct metadata was
extracted. But if an extract module produced invalid SPARQL that
would not be detected, until now.

Fixes tracker#196

When called with `--file --output-format=sparql`, it now correctly
separates the FileDataObject and the InformationElement resources.

While this is an API break, the API was added recently and I suspect
it's not used yet.

The old representation was a problem because the isStoredAs attribute,
which we use to relate files with their contents, was being used to
relate the bookmark to the website.

Old output:

    INSERT DATA {
      <urn:link:https:%2F%2Fen.wikipedia.org%2Fwiki%2FTracker_(search_software)> a nie:DataObject , nfo:Website , nfo:FileDataObject ;
        nie:url "https://en.wikipedia.org/wiki/Tracker_(search_software)" .
      _:1 nie:title "Wikipedia Tracker" ;
        nie:isStoredAs <urn:link:https:%2F%2Fen.wikipedia.org%2Fwiki%2FTracker_(search_software)> .
    };

New output:

    INSERT DATA {
      <file:///home/sam/src/tracker-miners/tests/functional-tests/test-extraction-data/desktop/link-wikipedia-tracker.desktop> a nfo:FileDataObject .
      <https://en.wikipedia.org/wiki/Tracker_(search_software)> a nie:DataObject , nie:Website ;
        nie:url "https://en.wikipedia.org/wiki/Tracker_(search_software)" .
      _:1 a nfo:Bookmark ;
        nie:isStoredAs <file:///home/sam/src/tracker-miners/tests/functional-tests/test-extraction-data/desktop/link-wikipedia-tracker.desktop> ;
        nie:title "Wikipedia Tracker" ;
        nie:bookmarks <https://en.wikipedia.org/wiki/Tracker_(search_software)> .
    };

See !181
for original MR.

tests: fix sched test under non glibc platforms

See merge request !275

This occured in the TestCase module, the "No test" is a placeholder
docstring which serves no purpose. By moving init later, we can pass
the name of the real test function, and there's no output.

As mentioned in tracker#170,
glibc's sched_setscheduler is rather broken and pthread functionality
should be used instead. musl only exposes a stub for sched_setscheduler
for this reason. tracker already correctly sets the scheduler via pthread,
but it didn't adjust the tests and as such the tests still failed on musl.

Extraction and other fixes

Closes #141

See merge request !273

The tracker-extract service is no longer deemed a "miner", but its
status is relevant to "tracker3 status" and "tracker3 daemon -f".
As tracker-extract is really an "implementation detail", proxy this
status via tracker-miner-fs, so these tools don't mislead if only
tracker-extract is active.

This is wrongly set on the nie:InformationElement, not on the
nfo:FileDataObject where it belongs. Causes SPARQL insertion errors for
files with chromaprint tags.

We were even testing for it wrong in the audio/flac-musicbrainz test,
update the expected output there.

Fixes: #141