Skip to content

Pluggable event loop backends (epoll support)

Submitted by Mikhail Zabaluev @mzabaluev

Link to original bug (#699132)

Description

An increasing number of modern application engines, such as Node.js and Rust, feature scalable event loop implementations such as libuv. There are other libraries that have their own implementations of an event loop, which an application developer might want to use as the event loop driver in preference to a GMainLoop, without needing to maintain a thread boundary. To interoperate with such engines and libraries efficiently, as well as to support modern scalable polling methods from the OS, the GLib event loop should support pluggable backend implementations.

Unfortunately, not only does GMainContext have a single compiled-in implementation, it's really designed around the old breed of poll(), select(), and WaitForMultipleObjects() on Win32. The main incompatibility point is that GPollFDs can be added with varying priorities which are honored on each iteration between prepare and poll, so that lower priority fd's do not get polled on if there is a higher priority non-fd source ready. This does not map well to the mechanics of event-based poll APIs.

After some early attempts (bug #156048) and more consideration, I think I have a feasible plan to add a new event loop API to GLib while preserving backwards compatibility with client code using GMainLoop/GMainContext.

  • Create a new class or interface GEventContext, which should become the base for event loop backends, built-in and third-party alike.
  • The interface for GEventContext and its usage by GSource should largely mirror that of GMainContext, except that all GPollFDs are considered to be of default priority.
  • For interoperability with old code, each GEventContext can provide an associated GMainContext, accessible as the default or thread default context parallel to the mother GEventContext's role. GMainContext is reimplemented to work in terms of the GEventContext backend, until...
  • When a client adds a GPollFD with a non-default priority to the GMainContext shim, the context switches into a compatibility mode, where all sources of the context are iterated in the old way by a separate thread which sends a wakeup to the context's event loop backend whenever there are sources ready. The separate thread is needed because the event backend may interoperate with other event sources unaccounted for in the GEventContext. There might be a log message to inform about the compatibility mode and its possible performance impact.
  • GLib gets built-in backends for poll, epoll, WaitForMultipleObjects, and whatever else is available on the supported platforms.
  • GMainContext can be deprecated as and when appropriate.

I'm going to start developing this as a feature branch, and create a glib-uv integration library as the reference third party event loop backend. If there are works in progress already or some other related experience, I'd love to hear about it in comments.