Skip to content

Merge gobject-introspection in GLib

The initial plan for gobject-introspection was to have it "incubate" as a separate repository until we figured out the kinks, and then merge it back into GLib. After 10+ years, the "kinks" have been largely figured out, and everyone depends on the introspection functionality one way or the other. Sadly, the merge never happened, because nobody had enough spare cycles to make it happen. This has caused various issues in the libgirepository API—especially when it comes to naming—and has produced what is, in essence, a circular dependency:

  • gobject-introspection depends on GLib, GObject, and GIO
  • gobject-introspection produces the introspection data for GLib, GObject, and GIO
  • documentation generators and API validation tools depend on the introspection data from GLib, GObject, and GIO
  • any change in GLib, GObject, and GIO must be propagated to gobject-introspection in order to actually exist from a tooling and binding perspective

What is gobject-introspection, really

GObject-introspection is composed of various moving parts

  • libgirepository, a library that loads typelib data, dlopens libraries, and figures out how to find and call symbols in those libraries
  • giscanner, a complex CPython, flexx/bison, Python module that
    1. scans C sources for declarations and C comments for annotations in order to build an AST of a C API
    2. generates, compiles, and runs an intermediate tool that inspects GTypes for inheritance, interfaces, properties, and signals, which can only be retrieved at run time
    3. transforms the basic XML representation of steps 1 and 2 into a more accurate ABI description using heuristics and annotations
    4. builds an XML representation of a GObject/C ABI
  • tools
    • g-ir-scanner: a tool that wraps the whole giscanner module
    • g-ir-compiler: a tool that turns the XML introspection data into a typelib

Additionally: giscanner contains a documentation generator (mainly used by GJS), and various other simple tools.

The goal

Being able to generate the introspection data for libglib, libgobject, and libgio when building GLib.

Being able to use the introspection data for generating the API reference for GLib.

Being able to provide a full reflection API to consumers of GLib.

Being able to improve the type system and its introspection at the same time.

The easy part

This is tracked as #3155 (closed).

Merging libgirepository is pretty quick. The library mainly needs some cleanups in the coding style and API for consistency, plus a pass for expanding the binary blob data structures that have expanded to their limit over the years, in order to give us some additional breathing room.

The changes are going to be ABI/API incompatible, so we will have to rename it to libgirepository-2.0, and we're going to need to rename the directory for the binary data to ${libdir}/girepository-2.0.

The XML data source does not require changes.

The hard part

Merging the GIR scanner and generator is going to be the biggest hurdle.

Adding a mixed CPython/Python module would add the Python development headers as a dependency to GLib, which may be problematic on smaller devices. We could add a build time option for environments that will never include non-C applications, and that would also remove the documentation build.

Alternatively, we could try and port giscanner to a pure Python implementation; that would require replacing the flex/bison C parser with an existing Python lexer capable of parsing C.

Finally, the most expensive alternative: porting giscanner to a pure C parser. The main issue is the creation of the AST representing the C API, and the overall parsing of C comments for documentation and annotations; that's a lot of code, encoding a lot of heuristics and a bunch of history.

Theoretically, we could split giscanner into a tool that does not depend on GLib at all. We could have GObject ship the small "type data dumper" binary, and tell g-ir-scanner to use it when generating introspection data; this would also remove the dependency on distutils, as we would not need to have g-ir-scanner build, compile, and run a small C utility. The remaining bits of giscanner that use GLib API can be turned into pure C, except for some GList/GHashTable use, which could be copy-pasted into giscanner itself, or replaced with libraries like c-util.

Edited by Philip Withnall