app: lower limits for selection operations

Select-> Grow/Shrink/Border can get criticals or crash GIMP when used on a large enough image, if the user sets radius too high. Max value is 2342px from app/operations/gimpoperation[border|grow|shrink].c. Couldn't find why that exact value is used, but I suppose some value had to be chosen.

These changes fix all crashes and criticals that I was able to find by lowering the allowed input. Created a 6000*6000 pixel image to be able to get into invalid values for all three operations.

Select Border was a special case depending on chosen border style, so lowered that by one more pixel to always work, without getting criticals. See comment in commit.

Example: Trace from GIMP crash on Select->Border

GNU Image Manipulation Program version 2.99.18
git-describe: GIMP_2_99_18
Build: org.gimp.GIMP.flatpak.dev rev 0 for linux
# C compiler #
Using built-in specs.
COLLECT_GCC=/usr/bin/cc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-unknown-linux-gnu/13.2.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: ../configure --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/bin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --enable-deterministic-archives --enable-shared --build=x86_64-bootstrapper-linux-gnu --host=x86_64-unknown-linux-gnu lt_cv_sys_lib_dlsearch_path_spec=/usr/lib/x86_64-linux-gnu --target=x86_64-unknown-linux-gnu --disable-multilib --enable-multiarch --disable-bootstrap --with-build-sysroot=/cross-installation --enable-languages=c,c++,fortran,objc,obj-c++ --enable-default-pie --enable-default-ssp --with-isl --disable-libssp --enable-linker-build-id --disable-libstdcxx-filesystem-ts --enable-cet host_configargs=lt_cv_sys_lib_dlsearch_path_spec=/usr/lib/x86_64-linux-gnu target_configargs=lt_cv_sys_lib_dlsearch_path_spec=/usr/lib/x86_64-linux-gnu --with-tune=generic
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 13.2.0 (GCC) 

# Libraries #
using babl version 0.1.108 (compiled against version 0.1.108)
using GEGL version 0.4.48 (compiled against version 0.4.48)
using GLib version 2.78.6 (compiled against version 2.78.4)
using GdkPixbuf version 2.42.12 (compiled against version 2.42.10)
using GTK+ version 3.24.42 (compiled against version 3.24.41)
using Pango version 1.51.2 (compiled against version 1.51.2)
using Fontconfig version 2.15.0 (compiled against version 2.15.0)
using Cairo version 1.18.0 (compiled against version 1.18.0)

# Flatpak info #
[Application]
name=org.gimp.GIMP
runtime=runtime/org.gnome.Platform/x86_64/45

[Instance]
instance-id=3913065798
instance-path=/home/anders/.var/app/org.gimp.GIMP
app-path=/home/anders/.local/share/flatpak/app/org.gimp.GIMP/x86_64/beta/e312204f4d207ad88accb3b4451be653b7388c9f86b84a6f2bc72ba2c734d669/files
app-commit=e312204f4d207ad88accb3b4451be653b7388c9f86b84a6f2bc72ba2c734d669
runtime-path=/home/anders/.local/share/flatpak/runtime/org.gnome.Platform/x86_64/45/730b4b38a4a285f7a8c995571d2b1d65414e5fb4d5dcdca6ac20cd7635960238/files
runtime-commit=730b4b38a4a285f7a8c995571d2b1d65414e5fb4d5dcdca6ac20cd7635960238
runtime-extensions=org.gnome.Platform.Locale=164c8e25aedc03f2296882a39b955e11f350587ba13b7746e0b22d2117dbfe90;org.freedesktop.Platform.GL.default=9981cd935556ef0ec146129d66b232fa1dc3b2f04eb83fe8e881cb64056d8c5f;org.freedesktop.Platform.openh264=bf24f23f3ba385f6e8c9215ed94d979db99814b0b614504a23a6d0751dc5f063;org.freedesktop.Platform.GL.default=1f6789acbc5410c18c9237f57ccbb34edda9b207429e6d74775e0aeb9fd66189
branch=beta
arch=x86_64
flatpak-version=1.14.10
session-bus-proxy=true
system-bus-proxy=true

[Context]
shared=network;ipc;
sockets=x11;wayland;fallback-x11;
devices=dri;
filesystems=xdg-config/GIMP;xdg-config/gtk-3.0;/tmp;xdg-run/gvfsd;host;xdg-run/gvfs;

[Session Bus Policy]
org.kde.kwin.Screenshot=talk
org.gtk.vfs.*=talk
org.gnome.Shell.Screenshot=talk
org.freedesktop.FileManager1=talk

[Environment]
ALSA_CONFIG_PATH=/usr/share/alsa/alsa-flatpak.conf
GI_TYPELIB_PATH=/app/lib/girepository-1.0
GST_PLUGIN_SYSTEM_PATH=/app/lib/gstreamer-1.0:/usr/lib/extensions/gstreamer-1.0:/usr/lib/x86_64-linux-gnu/gstreamer-1.0
XDG_DATA_DIRS=/app/share:/usr/share:/usr/share/runtime/share:/run/host/user-share:/run/host/share
ALSA_CONFIG_DIR=/usr/share/alsa
__EGL_EXTERNAL_PLATFORM_CONFIG_DIRS=/etc/egl/egl_external_platform.d:/usr/lib/x86_64-linux-gnu/GL/egl/egl_external_platform.d:/usr/share/egl/egl_external_platform.d
PYTHONUSERBASE=/var/data/python

fatal error: Segmenteringsfel

Stack trace:

/app/lib/libgimpbase-3.0.so.0(gimp_stack_trace_print+0x4be)[0x7f03082d97fe]
gimp-2.99(+0x2e2d8e)[0x55cd151e4d8e]
gimp-2.99(+0x2e33bc)[0x55cd151e53bc]
gimp-2.99(+0x2e602e)[0x55cd151e802e]
/usr/lib/x86_64-linux-gnu/libc.so.6(+0x3ee80)[0x7f030671fe80]
gimp-2.99(+0x310251)[0x55cd15212251]
/app/lib/libgegl-0.4.so.0(+0x87891)[0x7f0307bd8891]
/app/lib/libgegl-0.4.so.0(gegl_operation_process+0x180)[0x7f0307bddf90]
/app/lib/libgegl-0.4.so.0(gegl_graph_process+0x207)[0x7f0307be0347]
/app/lib/libgegl-0.4.so.0(gegl_eval_manager_apply+0xdb)[0x7f0307bdf3bb]
/app/lib/libgegl-0.4.so.0(gegl_node_blit+0xf8)[0x7f0307bc6a28]
gimp-2.99(gimp_gegl_apply_cached_operation+0x3b1)[0x55cd150df371]
gimp-2.99(gimp_gegl_apply_operation+0x29)[0x55cd150df8b9]
gimp-2.99(gimp_gegl_apply_border+0x240)[0x55cd150dfe20]
gimp-2.99(+0x1858be)[0x55cd150878be]
gimp-2.99(+0x48c283)[0x55cd1538e283]
/app/lib/libgimpwidgets-3.0.so.0(+0x5a5ff)[0x7f0307e5a5ff]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_closure_invoke+0x14a)[0x7f0307db443a]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x2c96c)[0x7f0307dc896c]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x2e3f1)[0x7f0307dca3f1]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit_valist+0x41)[0x7f0307dd03c1]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit+0x93)[0x7f0307dd0483]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x18652)[0x7f0307db4652]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x2e4af)[0x7f0307dca4af]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit_valist+0x41)[0x7f0307dd03c1]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit+0x93)[0x7f0307dd0483]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0xf5698)[0x7f0306cf5698]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x18652)[0x7f0307db4652]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x2e4af)[0x7f0307dca4af]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit_valist+0x41)[0x7f0307dd03c1]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit+0x93)[0x7f0307dd0483]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0xf377d)[0x7f0306cf377d]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0xadba3)[0x7f0306cadba3]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x18652)[0x7f0307db4652]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x2e4af)[0x7f0307dca4af]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit_valist+0x41)[0x7f0307dd03c1]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit+0x93)[0x7f0307dd0483]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0x1d40d3)[0x7f0306dd40d3]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_cclosure_marshal_VOID__BOXEDv+0x9a)[0x7f0307db7a9a]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x18652)[0x7f0307db4652]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x2e4af)[0x7f0307dca4af]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit_valist+0x41)[0x7f0307dd03c1]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit+0x93)[0x7f0307dd0483]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0x1d0e48)[0x7f0306dd0e48]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0x1d25e3)[0x7f0306dd25e3]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0x1d5aca)[0x7f0306dd5aca]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(gtk_event_controller_handle_event+0x73)[0x7f0306d97e43]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0x37c658)[0x7f0306f7c658]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0xa7137)[0x7f0306ca7137]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x18652)[0x7f0307db4652]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(+0x2d7b3)[0x7f0307dc97b3]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit_valist+0x41)[0x7f0307dd03c1]
/usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0(g_signal_emit+0x93)[0x7f0307dd0483]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0x37e194)[0x7f0306f7e194]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(+0x220e90)[0x7f0306e20e90]
/usr/lib/x86_64-linux-gnu/libgtk-3.so.0(gtk_main_do_event+0x696)[0x7f0306e22b26]
/usr/lib/x86_64-linux-gnu/libgdk-3.so.0(+0x42bd1)[0x7f0306b32bd1]
/usr/lib/x86_64-linux-gnu/libgdk-3.so.0(+0xa118e)[0x7f0306b9118e]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x5ed47)[0x7f0307cadd47]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x60e57)[0x7f0307cafe57]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(g_main_context_iteration+0x33)[0x7f0307cb0513]
/usr/lib/x86_64-linux-gnu/libgio-2.0.so.0(g_application_run+0x17d)[0x7f0307a23b7d]
gimp-2.99(app_run+0x1bb)[0x55cd151e489b]
gimp-2.99(main+0x3b7)[0x55cd150065b7]
/usr/lib/x86_64-linux-gnu/libc.so.6(+0x2808a)[0x7f030670908a]
/usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7f030670914b]
gimp-2.99(_start+0x25)[0x55cd15006745]