Crash while importing picture (from F-Spot db)
Submitted by an unknown user
Assigned to Lucas Beeler
Link to original bug (#718247)
Description
---- Reported by shotwell-maint@gnome.bugs 2011-10-11 23:20:00 -0700 ----
Original Redmine bug id: 4248
Original URL: http://redmine.yorba.org/issues/4248
Searchable id: yorba-bug-4248
Original author: Maxxer -
Original description:
While tring to import my fspot collection I ran into a crash.
I'm using Shotwell 0.11.2 in Ubuntu Oneiric.
The crash is attached, I'm not expert but I guess happens here:
#2 0x00007ffff41e54ad in g_assertion_message (domain=<optimized out>,
file=<optimized out>, line=<optimized out>,
func=0x7ffff7bc6be0 "gee_hash_map_node_iterator_next",
message=0x7fffc82e5000 "assertion failed: (self->_stamp == self->_map->priv->_stamp)")
at /build/buildd/glib2.0-2.30.0/./glib/gtestutils.c:1425
lstr = "2280\000\177\000\000\000\334\377\377\377\177\000\000p\362m\342\377\177 \000\000Pf\274\367\377\177\000"
s = 0x7fffc82de870 ""
#3 0x00007ffff41e59d2 in g_assertion_message_expr (domain=0x0,
file=0x7ffff7bc642c "hashmap.c", line=2280,
func=0x7ffff7bc6be0 "gee_hash_map_node_iterator_next", expr=) at /build/buildd/glib2.0-2.30.0/./glib/gtestutils.c:1436
s =
is that because of a unexpected string in comment/tag?
---- Additional Comments From shotwell-maint@gnome.bugs 2013-05-01 11:38:00 -0700 ----
History
Comment 1
Updated by Maxxer - about 2 years ago
- File shotwell.log.gz added
in shotwell.log:
L 3647 2011-10-12 03:38:38 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0005.JPG
L 3647 2011-10-12 03:38:38 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0006.JPG
L 3647 2011-10-12 03:38:38 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0008.JPG
L 3647 2011-10-12 03:38:39 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0009.JPG
L 3647 2011-10-12 03:38:39 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0010.JPG
L 3647 2011-10-12 03:38:40 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0011.JPG
L 3647 2011-10-12 03:38:41 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0012.JPG
L 3647 2011-10-12 03:38:41 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0013.JPG
L 3647 2011-10-12 03:38:42 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0014.JPG
L 3647 2011-10-12 03:38:42 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:42 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:42 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:42 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:42 [DBG] BatchImport.vala:1821: Importing /home/maxxer/Photos/2000/1/8/DSCF0015.JPG
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
L 3647 2011-10-12 03:38:43 [WRN] No namespace info available for XMP prefix `lr'
Comment 2
Updated by Maxxer - about 2 years ago
The last image in showtwell.log is here:
http://dl.dropbox.com/u/706934/DSCF0015.JPG
Comment 3
Updated by Adam Dingle about 2 years ago
- Priority changed from High to Urgent
- Target version set to 0.12
Comment 4
Updated by Maxxer - about 2 years ago
unlikely to be that pic causing the crash. I tried importing it in a temp install of shotwell master and was added to the library without issues. I also tried DSCF0016 and went fine as well. :(
Comment 5
Updated by Joseph Nuzman about 2 years ago
- File shotwell.gdb added
- File shotwell.log added
I'm hitting the same problem (assertion on import from f-spot db) with Shotwell 0.11.2 on Ubuntu 11.04 Natty x86-64 + Yorba PPA.
ERROR:hashmap.c:2322:gee_hash_map_node_iterator_next: assertion failed: (self->_stamp == self->_map->priv->_stamp)
I'm attaching my backtrace (looks more readable) and log as well.
During the postprocess_imported_media phase, an iterator is created to iterate over all the names in the Tag.global name_map. But sometime after the iterator is instantiated and before all the names have been indexed, the name_map is updated. The iterator catches this and triggers the assertion.
The option to watch the library dir for new files is not selected.
Comment 6
Updated by Joseph Nuzman about 2 years ago
-
File deleted (
_shotwell.log_)
Comment 7
Updated by Joseph Nuzman about 2 years ago
- File shotwell.log added
Comment 8
Updated by rv - about 2 years ago
Hi,
I have the same problem with 0.11.3.
Comment 9
Updated by Joseph Nuzman about 2 years ago
- File shotwell.gdb added
I also reproduced on 0.11.3, and managed to get a backtrace of the thread that modifies the name_map while the iterator is active (attached).
The culprit is in AlienDatabaseImportJob, in the "prepare" method.
if (detected_htags != null) {
Gee.Collection<string> paths = detected_htags.get_all_paths();
foreach (string path in paths)
Tag.for_path(path);
}
The Tag.for_path part updates the global tag map. If it happens while the other thread is iterating over the keys, the assertion is triggered.
Comment 10
Updated by Adam Dingle about 2 years ago
- Target version changed from 0.12 to 0.11.4
Thanks for all the investigation. We should consider fixing this for a new dot release 0.11.4.
Comment 11
Updated by Lucas Beeler about 2 years ago
- Category set to 4
- Status changed from Open to 5
- Assignee set to Lucas Beeler
- Resolution set to fixed
Fixed in 0add739d
Comment 12
Updated by Joseph Nuzman about 2 years ago
The proposed fix does not not work for me. I get the same assertion, with the same backtrace signature.
The proposed fix takes a read-only view of the collection before iterating over it. But as far as I can tell, the read-only view simply presents a restricted interface to the same underlying data structure. There is no duplication of the data structure or locking. So the other thread can still change the name_map, and the gee iterator code will still detect the inconsistent stamp and assert.
Comment 13
Updated by Maxxer - about 2 years ago
I tried importing this night as well, and didn't went thru. I imported less than 100 more pictures, but stopped pretty soon. I meant to run thru gdb this eve, but I guess it's useless right now. Thanks Joseph for your report, looking forward a new fix.
Comment 14
Updated by Adam Dingle about 2 years ago
- Status changed from 5 to Open
- Target version changed from 0.11.4 to 0.11.5
-
Resolution deleted (
_fixed_)
Unfortunately two users have reported that the fix in 0.11.4 does not work for them. Reopening for further investigation. If this is still not fixed, we should consider making yet another release 0.11.5.
Comment 15
Updated by rv - about 2 years ago
- File shotwell.log added
- File shotwell.gdb added
Still the same problem with 0.11.4, even earlier. Here are the backtraces
Comment 16
Updated by Bruno Girin about 2 years ago
I seem to remember when working on the F-Spot import first time round that Jim advised me that the threading model was such that some methods of BatchImportJob would be called over multiple threads while others would be called over a single thread. So it may be a case of moving the Tag.for_path call to a method that is called over a single thread. Maybe complete()? In fact, I'm not sure why there's any tag related code in prepare() considering that complete() seems to take care of it.
Does the reasoning make sense or have I missed something obvious?
Comment 17
Updated by Joseph Nuzman about 2 years ago
- File 0001-Protect-the-name_map-field-of-TagSourceCollection-fr.patch added
I've made a patch (attached) that demonstrates one way to resolve the issue. My approach was to add a GLib StaticMutex to protect readers/writers of Tag.global.name_map from concurrent access. Most of the readers and writers of this structure are internal to the TagSourceCollection class, and are easily guarded. The less elegant part is that an iterable view of the map is exposed externally using the get_all_names method. I therefore add explicit lock/unlock methods, which are used by the two callers of get_all_names to keep exclusive access while the map is iterated over.
My patch protects only the name_map field, and I think the threading model should be reviewed to make sure things are otherwise sane. But import of several thousand pictures completed successfully after this fix.
Comment 18
Updated by Adam Dingle about 2 years ago
- Status changed from Open to Review
Thanks for the patch! Lucas, please review.
Comment 19
Updated by Jan Moren about 2 years ago
I had this bug - trying to import about 10k images from f-spot when I updated to Ubuntu 11.10. I'd get a few images imported followed by this error.
I'm happy to say that the 11.5 version happily imported all of my images, so the bug seems to be fixed.
Comment 20
Updated by Adam Dingle about 2 years ago
- Status changed from Review to 5
- Resolution set to fixed
Fixed in 0.11.5. Closing.
Comment 21
Updated by Charles Lindsay 7 months ago
- Status changed from 5 to Fixed
--- Bug imported by chaz@yorba.org 2013-11-25 21:55 UTC ---
This bug was previously known as bug 4248 at http://redmine.yorba.org/show_bug.cgi?id=4248 Imported an attachment (id=262269) Imported an attachment (id=262270) Imported an attachment (id=262271) Imported an attachment (id=262272) Imported an attachment (id=262273) Imported an attachment (id=262274) Imported an attachment (id=262275) Imported an attachment (id=262276)
Unknown Component Using default product and component set in Parameters Unknown milestone "unknown in product shotwell. Setting to default milestone for this product, "---". Setting qa contact to the default for this product. This bug either had no qa contact or an invalid one.
Version: 0.11.5
Resolution: RESOLVED FIXED