Skip to content

Improve insert performance

Carlos Garnacho requested to merge wip/carlosg/insert-perf into master

This branch does some different things:

  • Removes the ontology vs regular resource ID distinction. At the expense of some operations that used to distinguish/preserve those (e.g. DROP GRAPH DEFAULT, CLEAR GRAPH DEFAULT) not being able to do so anymore. This unlocks some faster behavior of resource insertions, and makes us look better wrt potential ROWID overflows.
  • Improves insertion of new resource URIs
    • Avoids selects before insertion if possible
    • Caches newly inserted resources better, to avoid old data/type lookups
  • Avoids some string conversions and copies when executing TrackerSparqlStatement queries

This avoids many interspersed SELECTs in the regular course of processing the insertion of new resources. Brings the benchmark test at !345 (merged) up to ~750K insertions in 30s, and brings tracker-miner-fs-3 first index a little bit further down.

Merge request reports