Skip to content

Use posix_spawn for optimized process launching

Daniel Drake requested to merge dsd/glib:master into master

When the amount of free memory on the system is somewhat low, gnome-shell will sometimes fail to launch apps, reporting the error:

fork(): Cannot allocate memory

fork() is failing here because while cloning the process virtual address space, Linux worries that the thread being forked may end up COWing the entire address space of the parent process (gnome-shell, which is memory-hungry), and there is not enough free memory to permit that to happen.

In this case we are simply calling fork() in order to quickly call exec(), which will throw away the entirity of the duplicated VM, so we should look for ways to avoid the overcommit check.

The well known solution to this is to use clone(CLONE_VM) or vfork(), which completely avoids creating a new memory address space for the child. However, that comes with a bunch of caveats and complications:

In 2016, glibc's posix_spawn() was rewritten to use this approach while also resolving the concerns.;h=9ff72da471a509a8c19791efe469f47fa6977410

I experimented with a similar approach in glib, but it was not practical because glibc has several items of important internal knowledge (such as knowing which signals should be given special treatment because they are NPTL implementation details) that are not cleanly exposed elsewhere.

Instead, this series adapts the gspawn code to use posix_spawn() where possible, which will reap the benefits of that implementation. It also exposes the required functionality through gdesktopappinfo so that the shell can use it.

In addition to allowing for the gnome-shell "Cannot allocate memory" failure to be avoided, this should result in a general speedup in this area, because fork()'s behaviour of cloning the entire VM space has a cost which is now avoided. posix_spawn() has also recently been optimized on OpenSolaris as the most performant way to spawn a child process.

Edited by Daniel Drake

Merge request reports