GitLab

We don't deal with media art in tracker-extract since commit
6a050686 and ac84484d, mostly as it would require
us to weaken the tracker-extract sandbox.

The tracker_extract_get_metadata() function should return FALSE if
an error occurs reading the file. The tracker-extract-text module would
return TRUE in all cases.

This was causing intermittent failures in the functional-tests, as the
following situation could occur:

  1. file2.txt is created
  2. tracker-miner-fs sees file2.txt and processes it
  3. file2.txt is deleted (or moved into an unmonitored directory)
  4. tracker-miner-fs sees the deletion and removes its resource from
     the store
  5. tracker-extract sees the created notification for file2.txt and
     tries to process it
  6. the tracker_extract_get_metadata() incorrectly returns TRUE
     (success), so tracker-extract recreates the deleted resource

This problem was being detected in the functional tests and was
causing intermittent failures.

This hopefully fixes: GNOME/tracker-miners#56

Properly handle (atend) when parsing Postscript file. Also add a test
case.

Process ps.gz files using GZlibDecompressor instead of spawing gunzip. Because
tracker-extract runs the file parsers inside a seccomp sandbox, spawning an
external process during parsing is not a good idea because it leaves us with
little control on which syscalls are used.

Closes: GNOME/tracker-miners#61

This is the most generic class that still triggers GraphUpdated, we need
to delegate on the extractor module to figure out whether the file is a
music piece on its own or a container of music pieces.

In the latter case the role belongs on the contained elements, not on the
file resource itself, so avoid setting it here for all.

If there is a TOC, the nmm:* information belongs in the child nmm:MusicPiece
elements, not on the container file.

The migration to TrackerResource broke the handling of flac/cue, as
tracker_resource_print_*() do expect all related resources to be directly
referenced from the root resource, but the gstreamer extractor just added
backreferences from the nmm:MusicPieces to the parent nfo:FileDataObject.

Add a relation both ways through nie:hasLogicalPart/nie:isLogicalPartOf
so the graph is complete, and the tracks referenced from the root element.

Closes: GNOME/tracker-miners#60

Closes: GNOME/tracker-miners#59

Allow running tests without the need to install the gsettings schemas.
This commit moves all the gschemas in the data directory and compiles
them in the build directory. Tests that require gschemas need to be
launched with the proper GSETTINGS_SCHEMA_DIR env variable.

Based on tracker.git commit c33c080c20dae1e60982, by Andrea Azzarone.

GSettings schema files are not supposed to include translations,
but rather tools should look up translations in .mo files.

Fixes https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=918122

Fixes GNOME/tracker-miners#15 and
GNOME/tracker-miners#40.

This is the first step towards folding tracker-miner-apps into
tracker-miner-fs, thus dropping one executable from a usual session.

Implement an extractor for .desktop files as per the desktop entry
specification [1]. This is basically a rip off of src/miner/apps,
just tuned down to ignore directory/link types, and made to use
TrackerResource instead of the deprecated TrackerSparqlBuilder.

[1] https://specifications.freedesktop.org/desktop-entry-spec/latest/

This reverts commit ab29a9cd.

The name watch was correct in the first place. This also
reverts the two cleanup attempts f9e7de0f and 8b6bc5b7.

This fixes a regression from 8b6bc5b7.

The extractor rules defined an array inside an array, which configure_file()
wouldn't accept.

Commit ab29a9cd got it wrong. This is not a tracker-store watch,
but a watch on the domain ontology owner. It should only apply when
there is one such.

The tracker_resource_add_* API does not delete the previous property value,
so may result in constraint errors if these values happen to change (some
are more likely than others).

This commit fixes this entire source of bugs en-masse, the affected
properties were:

- nmm:albumDiscAlbum
- nfo:genre in GStreamer extractor
- nie:license in HTML extractor
- nco:publisher and nfo:equipment in JPEG extractor
- nfo:equipment in RAW extractor

Using tracker_resource_set_relation() ensures we respect it.

Closes: #20

Instead of looking up directly GST_TAG_DATE_TIME, use the same extraction
paths than nie:contentCreated. It might fall back into GST_TAG_DATE, which
may be a mismatch.

Commit dd5f39ab fixed nfo:orientation, but the other nmm:* fields
as extracted from XMP data can cause the same problems as the PDF
document is not a nmm:Photo.

Related: tracker#21

Currently Tracker domain rules must be installed inside Tracker's data
directory (usually /usr/share/tracker). This is limiting as it means
only system packages can add them. A program installed into /opt is
unable to use a custom domain, for example. Since Tracker is implemented
as a system of daemons, it's not particularly straight forward to work
around this by setting XDG_DATA_DIRS= to point somewhere non-standard
either.

This patch removes this restriction in a simple way: it allows users to
pass a full path to the domain rule, rather than just the base name.

We should be consistent about what the name of a Tracker domain actually
is. In the domain rule we specify Domain=org.freedesktop (for the
default rule) or Domain=org.example.App (for a custom domain). But
internally tracker_domain_ontology_get_domain() would return
org.freedesktop.Tracker1 or org.example.App.Tracker1, i.e. the domain
name now has '.Tracker1' appended.

This does make sense in most cases, but it means there's no way to get
the actual name of the domain from a TrackerDomainOntology object. This
commit changes the existing function to not append '.Tracker1' in the
value it returns, which means that function can now be used to get the
base name of the domain. The assumption is that callers are normally
appending stuff to this base name anyway so it's not much extra effect
to also append the '.Tracker1' component if needed.

libtracker-common is internal to Tracker so this doesn't constitute a
public API break.

We try to format the timezone hour difference with %02d, however for
negative numbers it will contain just eg. "-4", which is not the expected
behavior.

So, take the negative sign out of the integer, and add it separately
just like we do for the explicit '+'. This makes the %02d bit actually
correct, so the timezone bit starts with "-04".

GNOME/tracker#22

Closes: tracker#22

The nfo:orientation property can only be set on nfo:Image resources, not
nfo:PaginatedTextDocument resources.

Fixes GNOME/tracker#21

Whatever PopplerDest.named_dest is, it's not a real string. This triggers
UTF-8 warnings on any PDF file with such actions.

GNOME/tracker-miners#12

Closes: #12

Some image metadata formats (Exif, Iptc) don't have encoding
guarantees, so do our best here.

GNOME/tracker-miners#13

Closes: #13

Some image metadata formats (Exif, Iptc) don't have encoding
guarantees, so do our best here.

GNOME/tracker-miners#13

Closes: #13

The TrackerExtractPersistence stuff takes care of marking files as ignored
whenever those trigger crashes. However for the files that simply error out
we just add those to a runtime blacklist, which gets forgotten on next runs.

Those will trigger message/warning logging each time tracker-extract is
restarted, which is more often now. Just mark those as failures as it'd
happen with files resulting in crashes. We need to think how these files
are generically retried after extractor fixes/updates, but that is also
missing for crashing files.

GNOME/tracker-miners#6

Closes: #6

In order to run tracker-extract without installing it, we set the
TRACKER_EXTRACT_RULES_DIR path to point to the src/tracker-extract
directory.

This is problematic because that directory contains lots of stuff
that isn't .rule files and also contains .rule files for extractors
which might not be enabled. The result being pointless warnings
during the tests like this:

    ** (tracker-extract:29193): WARNING **: 16:18:11.703: Could not load module '/home/sam/src/tracker-miners/build/tests/functional-tests/../../src/tracker-extract/libextract-libav.so': /home/sam/src/tracker-miners/build/tests/functional-tests/../../src/tracker-extract/libextract-libav.so: cannot open shared object file: No such file or directory

Using Meson it's fairly straightforward to create a separate
directory in the build tree to contain these files, which fixes
the warnings.

The same thing could be implemented for Autotools, but it doesn't
affect the actual functionality of the tests so I haven't spent time
on it myself.

Fixes GNOME/tracker-miners#9

This depends on GNOME/tracker!3

This fixes the following errors from tracker-extract:

    tracker_extract_new_contact: assertion 'fullname != NULL' failed
    tracker_resource_add_relation: NULL is not a valid value.
    g_object_unref: assertion 'G_IS_OBJECT (object)' failed

We were ignoring the error from g_option_context_parse() here, which
would lead to weird behaviour further on.