Crash when flattening an XCF file
Submitted by aferrero1975
Link to original bug (#796317)
Description
Created attachment 372317 sample file to reproduce the crash
I am testing GIMP under MacOS 10.10.5 (XCode 6.3 + dependencies installed via homebrew). I was able to reproduce a crash originally reported in this post: https://discuss.pixls.us/t/nightly-gimp-macos-builds-testing-needed/7765/9
The crash occurs when the attached .xcf file is opened and then immediately flattened. With the debugger I could trace the crash down to this line of code: https://git.gnome.org/browse/gimp/tree/app/operations/layer-modes/gimpoperationlayermode.c?h=gimp-2-10#n642
However, I could not yet figure out what is exactly wrong with the code, and provide a patch. Nevertheless, if anyone comes out with a fix I'll be glad to checks it.
The crash has been reported on MacOS, but I have the impression that it is more general.
Here is the obtained stack backtrace:
Process 22738 stopped
* thread #28: tid = 0x13b1a8, 0x00000001003ade30 gimp`gimp_operation_layer_mode_real_process(operation=0x000000010d7f7b00, in_p=<unavailable>, layer_p=<unavailable>, mask_p=0x0000000000000000, out_p=<unavailable>, samples=<unavailable>, roi=<unavailable>, level=<unavailable>) + 640 at gimpoperationlayermode.c:642, name = 'pool', stop reason = EXC_BAD_ACCESS (code=2, address=0x108f38d8c)
frame #0: 0x00000001003ade30 gimp`gimp_operation_layer_mode_real_process(operation=0x000000010d7f7b00, in_p=`<unavailable>`, layer_p=`<unavailable>`, mask_p=0x0000000000000000, out_p=`<unavailable>`, samples=`<unavailable>`, roi=`<unavailable>`, level=`<unavailable>`) + 640 at gimpoperationlayermode.c:642
639 */
640 while (i < end && (in[i] == 0.0f || layer[i] == 0.0f))
641 {
-> 642 blend_out[i] = 0.0f;
643 i += 4;
644 }
645
thread #30: tid = 0x13b1e2, 0x00000001003ade30 gimp`gimp_operation_layer_mode_real_process(operation=0x000000010d7f7b00, in_p=<unavailable>, layer_p=<unavailable>, mask_p=0x0000000000000000, out_p=<unavailable>, samples=<unavailable>, roi=<unavailable>, level=<unavailable>) + 640 at gimpoperationlayermode.c:642, name = 'pool', stop reason = EXC_BAD_ACCESS (code=2, address=0x109700d8c)
frame #0: 0x00000001003ade30 gimp`gimp_operation_layer_mode_real_process(operation=0x000000010d7f7b00, in_p=`<unavailable>`, layer_p=`<unavailable>`, mask_p=0x0000000000000000, out_p=`<unavailable>`, samples=`<unavailable>`, roi=`<unavailable>`, level=`<unavailable>`) + 640 at gimpoperationlayermode.c:642
639 */
640 while (i < end && (in[i] == 0.0f || layer[i] == 0.0f))
641 {
-> 642 blend_out[i] = 0.0f;
643 i += 4;
644 }
645
thread #31: tid = 0x13b1e3, 0x00000001003ade30 gimp`gimp_operation_layer_mode_real_process(operation=0x000000010d7f7b00, in_p=<unavailable>, layer_p=<unavailable>, mask_p=0x0000000000000000, out_p=<unavailable>, samples=<unavailable>, roi=<unavailable>, level=<unavailable>) + 640 at gimpoperationlayermode.c:642, name = 'pool', stop reason = EXC_BAD_ACCESS (code=2, address=0x109d6fd8c)
frame #0: 0x00000001003ade30 gimp`gimp_operation_layer_mode_real_process(operation=0x000000010d7f7b00, in_p=`<unavailable>`, layer_p=`<unavailable>`, mask_p=0x0000000000000000, out_p=`<unavailable>`, samples=`<unavailable>`, roi=`<unavailable>`, level=`<unavailable>`) + 640 at gimpoperationlayermode.c:642
639 */
640 while (i < end && (in[i] == 0.0f || layer[i] == 0.0f))
641 {
-> 642 blend_out[i] = 0.0f;
643 i += 4;
644 }
645
(lldb) bt
* thread #28: tid = 0x13b1a8, 0x00000001003ade30 gimp`gimp_operation_layer_mode_real_process(operation=0x000000010d7f7b00, in_p=<unavailable>, layer_p=<unavailable>, mask_p=0x0000000000000000, out_p=<unavailable>, samples=<unavailable>, roi=<unavailable>, level=<unavailable>) + 640 at gimpoperationlayermode.c:642, name = 'pool', stop reason = EXC_BAD_ACCESS (code=2, address=0x108f38d8c)
* frame #0: 0x00000001003ade30 gimp`gimp_operation_layer_mode_real_process(operation=0x000000010d7f7b00, in_p=`<unavailable>`, layer_p=`<unavailable>`, mask_p=0x0000000000000000, out_p=`<unavailable>`, samples=`<unavailable>`, roi=`<unavailable>`, level=`<unavailable>`) + 640 at gimpoperationlayermode.c:642
frame #1: 0x00000001010fbc5a libgegl-0.4.0.dylib`thread_process(thread_data=0x00007fff5fbfd0a0, unused=<unavailable>) + 314 at gegl-operation-point-composer3.c:84
frame #2: 0x0000000101518e2a libglib-2.0.0.dylib`g_thread_pool_thread_proxy + 37
frame #3: 0x00000001015181b3 libglib-2.0.0.dylib`g_thread_proxy + 90
frame #4: 0x00007fff954df05a libsystem_pthread.dylib`_pthread_body + 131
frame #5: 0x00007fff954defd7 libsystem_pthread.dylib`_pthread_start + 176
frame #6: 0x00007fff954dc3ed libsystem_pthread.dylib`thread_start + 13
Attachment 372317, "sample file to reproduce the crash":
field.xcf
Version: 2.10.2