Gdk-CRITICAL creating shared memory file (using memfd_create) failed: Too many open files makes QEMU 5.2 to crash
Steps to reproduce
Using QEMU 5.2 in the host machine with the followings arguments:
/usr/bin/qemu-system-x86_64 -monitor stdio -device AC97 -k it -machine accel=kvm -m 6144 -drive file=/home/andrea/Kali,media=disk -boot once=c,menu=off -net nic -net user,hostfwd=tcp::9999-:22 -rtc base=localtime -name "Kali Linux" -vga virtio -cpu host -smp 2
to run a Kali Linux guest machine, with default packages all updated to the last version available.
Running nmap -sT -sV 192.168.1.1 in the guest machine, raises a Gdk-CRITICAL error and makes QEMU to crash with a segmentation fault.
Already reported to the QEMU group, the issue is related to the Slirp library used by QEMU, Slirp itself doesn't crash, but it exhausts all available FDs wrt ulimits, and then causes GTK to crash.
A Slirp issue report has already been open to its manteiners.
GTK issue analysis by a RedHat engineer if can be useful:
in GTK source I see memfd_create called from open_shared_memory().
This handles failure by printing this error message and returning -1.
The caller is create_shm_pool(). This checks for -1 from open_shared_memory and returns NULL.
The caller is _gdk_wayland_display_create_shm_surface(). This dos NOT check for NULL, and this I expect this is what eventually results in the crash when a NULL pointer is accessed.
Version information
GTK version 3.24.24-1 on Kali Linux
Warnings
(qemu:10209): Gdk-CRITICAL **: 15:04:52.279: ../../../../../gdk/wayland/gdkdisplay-wayland.c:1299: creating shared memory file (using memfd_create) failed: Too many open files
Backtrace
Thread 1 "qemu-system-x86" received signal SIGSEGV, Segmentation fault.
0x00007ffff2da9274 in wl_proxy_marshal_constructor () from /lib/x86_64-linux-gnu/libwayland-client.so.0
(gdb) backtrace
#0 0x00007ffff2da9274 in wl_proxy_marshal_constructor () from /lib/x86_64-linux-gnu/libwayland-client.so.0
#1 0x00007ffff36d757e in ?? () from /lib/x86_64-linux-gnu/libgdk-3.so.0
#2 0x00007ffff36ccd06 in ?? () from /lib/x86_64-linux-gnu/libgdk-3.so.0
#3 0x00007ffff3673ce0 in gdk_cursor_new_from_pixbuf () from /lib/x86_64-linux-gnu/libgdk-3.so.0
#4 0x00007ffff7fc9654 in ?? () from /usr/lib/x86_64-linux-gnu/qemu/ui-gtk.so
#5 0x0000555555acd692 in dpy_cursor_define ()
#6 0x0000555555a9d720 in ?? ()
#7 0x0000555555a9daad in ?? ()
#8 0x0000555555de1765 in aio_bh_poll ()
#9 0x0000555555dc8c8e in aio_dispatch ()
#10 0x0000555555de164e in ?? ()
#11 0x00007ffff703ff2e in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#12 0x0000555555ddaef0 in main_loop_wait ()
#13 0x0000555555c96869 in qemu_main_loop ()
#14 0x000055555591704e in main ()