GitLab

Fix some SPARQL corner cases

Closes #259

See merge request !377

As the group_concat ordering is undefined, test with just one value.

Do not convert results to string except in the topmost property function,
so things like nmm:artistName(nmm:performer(?u)) work. This allows arbitrary
nesting of single and multivalued properties to work consistently.

Fixes: #259

Test that the service graph pattern comes up empty if there is
no pattern/variables to project.

Querying a service that does not use any variable in its graph pattern
is fairly pointless. There are no variables to project into the rest
of the query, thus there's no way to match anything with the service
graph pattern. In pure SPARQL, that is basically just:

  SELECT * { SERVICE <...> { } }

But since in Tracker we are more lenient wrt giving each projected
variable a name, we also have cases like:

  SELECT * { SERVICE <...> { SELECT 2 { } } }

That are anyways pointless to execute. Detect all these cases and
send an empty query string instead, the service vtable will behave
as expected.

Make these queries succeed, but come up empty. This is effectively the
same than asking the remote service for the empty graph pattern "{ }".

Test all combinations and orders of these 2 modifiers.

In Sparql, OFFSET and LIMIT can be set independently (and event flip
around!), however SQL mandates that OFFSET also needs LIMIT. Handle
correctly the case that only OFFSET is given, by setting an unlimited
limit.

We currently happen to be needlessly regenerating the SQL string from
the SPARQL query, as the TrackerSparql deems to need one as the generation
number is not up-to-date.

We can safely assign the generation number on construction, before the
first parsing.

Ensure these work correctly.

We currently don't handle correctly the situation of GRAPH ?g { ... }
clauses in SELECTs that leave ?g unbound. E.g. situations that the
contained GroupGraphPattern does not contain anything that actually
requires accessing graphs:

  SELECT ?g { GRAPH ?g { } }
  SELECT ?g ?a { GRAPH ?g { BIND (1 AS ?a) } }

Check for this specific situation, and union the contained query
with a query for all available named graphs. This makes these queries
behave as expected (e.g. bind ?g to all available graphs, because
the empty pattern { } matches everything).

This is allowed as long as the variable was not previously defined
in the same group graph pattern, forbidden otherwise. Test both
cases.

As per https://www.w3.org/TR/sparql11-query/#bind:
"The variable introduced by the BIND clause must not have been
used in the group graph pattern up to the point of use in BIND."

We are currently checking that the variable has bindings, but
that's also true if the variable was used in other graph patterns
prior to the current one.

We must check here only in the current context to match that
behavior. This makes queries like:

  SELECT ?u { { BIND (1 AS ?u) } UNION { BIND (2 AS ?u) } }

work as expected.

Fixes to change notifications in the sandbox

Closes #287

See merge request !373

Propagate all errors in tracker-data-update.c

Closes tracker-miners#150

See merge request !376

libtracker-sparql: Allow prefixed names as per spec in TrackerResource

Closes #286

See merge request !375

libtracker-data: Add an usage reference early when getting an interface

Closes tracker-miners#154

See merge request !374

Propagate any errors that might happen in this function.

These errors are just warned in place, propagate them up.

These errors are ignored here. Ensure they are propagated upwards.

Errors were ignored in some situations, ensure they are propagated
properly.

There's several things happening within that don't get errors properly
propagated, add a GError argument all the way through.

These are just warned in place, propagate them further up.

Instead of initializing the modseq from the database at a random point
without possibility of error propagation, ensure this is initialized when
beginning the first non-ontology transaction, and propagate the possible
error.

This allows the rest of the modseq code be oblivious about GErrors
without shame.

Instead of keeping errors for itself, propagate those up, all the way
up.

Fixes: tracker-miners#150

Reorder a bit the code, and don't account for NULL subject which
must not happen here.

In order to figure out whether TrackerResource is dealing with a prefixed
name, we used g_uri_parse_scheme(). This happens to work for the most
common chars used in prefixes, however there's a substantial difference in
the charset allowed. For URI schemes (from
https://tools.ietf.org/html/rfc3986):

  scheme      = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )

For prefixed name prefixes (from
https://www.w3.org/TR/sparql11-query/#sparqlGrammar):

  PNAME_NS	  ::=  	PN_PREFIX? ':'
  PN_PREFIX	  ::=  	PN_CHARS_BASE ((PN_CHARS|'.')* PN_CHARS)?
  PN_CHARS_BASE	  ::=  	[A-Z] | [a-z] | [#x00C0-#x00D6] | [#x00D8-#x00F6] |
                        [#x00F8-#x02FF] | [#x0370-#x037D] | [#x037F-#x1FFF] |
                        [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] |
                        [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] |
                        [#x10000-#xEFFFF]
  PN_CHARS_U	  ::=  	PN_CHARS_BASE | '_'
  PN_CHARS	  ::=  	PN_CHARS_U | '-' | [0-9] | #x00B7 | [#x0300-#x036F] |
                        [#x203F-#x2040]

Even though it's a bit of a layering break, lean on the SPARQL parser for
parsing prefixes exactly as per the spec.

Fixes: #286

Currently, there's a gap between a TrackerDBInterface being picked for
execution, and a cursor increasing its usage counter. This opened the
following race condition:

 - Random thread executes SPARQL select query, a TrackerDBInterface
   is picked.
 - TrackerSparqlConnection reaches a threshold/timeout and tries to
   release memory, performed by the update thread
 - All unused readonly connections are closed
 - The random thread proceeds to create a stmt/cursor, finds invalid
   memory in place of the TrackerDBInterface
 - Whoops

To fix this, increase the usage counter when getting a readonly
interface from the TrackerDBManager, while the connection pool is
locked (thus mutually exclusive with the "release memory" operation)
and decrease it after having created the statement/cursor (at which
point it is additionally increased by the cursor).

This will ensure that an interface is considered "in use" as soon
as it gets out of the readonly connections pool, usage counter will
always be greater than 1 until there are no cursors left.

Fixes: tracker-miners#154

We'll want additional users of this, so generalize the
tracker_db_interface_get_is_used() accounting and make it internal
API.

Test that implicit graph name expansion is properly handled.

Implicitly expand the graph URI, in case we are given a prefixed name.

The propagation of events to a local TrackerSparqlConnection used
to be broken. Add a test to ensure we don't regress here.