Shotwell is unresponsive with big photo library
Submitted by an unknown user
Link to original bug (#718786)
Description
---- Reported by shotwell-maint@gnome.bugs 2012-10-07 14:32:00 -0700 ----
Original Redmine bug id: 5972
Original URL: http://redmine.yorba.org/issues/5972
Searchable id: yorba-bug-5972
Original author: jf c
Original description:
Hi,
At opening, Shotwell browse the photo library. The log reports: Directory loop detected at... Same directory is mentioned several times.
Then CPU is very high, and user interface is unresponsive.
The program doesn't crash. From time to time, it take user actions.
This make this software unusable.
Note: the option "Watch library directory" is not set.
See screen shot.
Context:
40000+ photos, 4000+ directories. The library is placed on a NAS mounted with
NFS over a gigabyte network.
The NAS reports no errors on its filesystem.
Workstation : Dell 6520 Ubuntu 12.04 64bits, 8GB RAM
Shotwell 0.13.1, Problem seen also with previous versions
Fell free to ping me.
JFC
Related issues:
- related to shotwell - 4834: Cannot use NFS share as library (Open)
- related to shotwell - 2412: Shotwell crash while importing from network drive (Invalid)
- related to shotwell - Feature #3130 (closed): Disable runtime monitoring (don't scan library directory ... (Open)
---- Additional Comments From shotwell-maint@gnome.bugs 2013-05-16 14:35:00 -0700 ----
History
Comment 1
Updated by Jim Nelson about 1 year ago
- Target version set to 0.14.0
I suspect this problem has to do with symbolic links. The directory monitor attempts to locate loops in the directory tree that would cause it to descend forever.
Are any of these directories symbolic links to elsewhere in your pictures directory?
Comment 2
Updated by jf c about 1 year ago
Thanks Jim,
I have checked the library for symbolic links (find . -type l
). No symbolic
links at all.
Comment 3
Updated by Jim Nelson about 1 year ago
Then that means GVFS is returning the same File ID for two different directories: http://developer.gnome.org/gio/2.30/GFileInfo.html#G-FILE- ATTRIBUTE-ID-FILE:CAPS This is a problem as that's how Shotwell's directory monitor detects symlinked files and directories.
We've had complaints in the past about Shotwell working with NAS and network drives. It's something we've been meaning to attack for a while. My concern is that NFS does not support File ID, which is important for directory monitoring to work.
Comment 4
Updated by jf c about 1 year ago
Jim,
I don't use GVFS.
Directory monitoring is desirable feature, but I have switched off.
May be the question is why Showtwell is crawling the library even if the option is of.
I would like to keep the NAS for many advantages.
I'll check with an attached HD when I'll have some free time.
Comment 5
Updated by Jim Nelson about 1 year ago
Shotwell does scan your library at startup to detect missing files. There is a ticket to remove this: #4754 (closed)
Comment 6
Updated by jf c about 1 year ago
Jim,
I have copied all my photos to an external ext4 disk. I haven't seen any freezing. Then, NFS seems to be in cause in my problem.
Then I have tried to use my NAS again. But I have messed up my settings and I have lost my database. Apparently, there are some problems when using multiple library. Thumbnails become completely mixed up. But this is out of this topic.
Finally I have erased all settings, and start over a new database using my NAS. No freeze has happened so far. Then NFS is not the problem...
In log file, I can see some access on the NAS despite the fact the "Watch library" option is un-ticked.
Any idea?
Comment 7
Updated by jf c about 1 year ago
Hi Jim,
This morning, Shotwell is freezing again. The NAS hasn't changed over the night, the files structure is remained identical...
Here are facts:
-
Importing from NFS directory is not a problem.
-
During one day, Showtwell was very responsive
This morning: -
Opening Showtwell is fast.
-
Freeze comes when changing of event:
L 5429 2012-10-14 10:03:00 [WRN] DirectoryMonitor.vala:842: Directory loop detected at /net/nfs/qnap/Photos/JPGImages/2005, not exploring
...
...
Note directory that comes first in the log is not related to the event I have clicked on.
Could you check why Directory Monitor is invoked when switching of event and "Watch directory" option if off?
Don't hesitate to ping me.
++ jfc
Comment 8
Updated by Jim Nelson about 1 year ago
So it does sound like NAS is the problem. I assume you let Shotwell run on ext4 for some time without an issue.
Although Watch Directory is turned off, Shotwell does do a startup scan to see if any files have been moved or deleted since its last run. This scan happens only once, when Shotwell starts. There's a ticket we're considering to allow the user to turn this off (#4754 (closed)).
Unfortunately, I think the only solution right now is not to use NAS for your photo library. This is definitely something we want to attack in 0.14.
Comment 9
Updated by jf c about 1 year ago
Hi Jim,
At the moment this is temporary, I can use the external drive I use
usually for saving the NAS... So my backup schema must be changed.
I'm looking forward for a Watch Directory feature that works on NAS, NTFS.
++ JFC
2012/10/15 redmine@redmine.yorba.org:
Issue #5972 has been updated by Jim Nelson.
So it does sound like NAS is the problem. I assume you let Shotwell run on ext4 for some time without an issue.
Although Watch Directory is turned off, Shotwell does do a startup scan to see if any files have been moved or deleted since its last run. This scan happens only once, when Shotwell starts. There's a ticket we're considering to allow the user to turn this off (#4754 (closed)).
Unfortunately, I think the only solution right now is not to use NAS for your photo library. This is definitely something we want to attack in 0.14.
5972: Shotwell is unresponsive with big photo library
http://redmine.yorba.org/issues/5972#change-19326
Author: jf c
Status: Open
Priority: High
Assignee:
Category:
Target version: 0.14
Resolution:
Keywords:
Hi,
At opening, Shotwell browse the photo library. The log reports: Directory loop detected at... Same directory is mentioned several times.
Then CPU is very high, and user interface is unresponsive.
The program doesn't crash. From time to time, it take user actions.
This make this software unusable.
Note: the option "Watch library directory" is not set.
See screen shot.
Context:
40000+ photos, 4000+ directories. The library is placed on a NAS mounted with NFS over a gigabyte network.
The NAS reports no errors on its filesystem.
Workstation : Dell 6520 Ubuntu 12.04 64bits, 8GB RAM
Shotwell 0.13.1, Problem seen also with previous versions
Fell free to ping me.
JFC
--
You have received this notification because you have either subscribed to it, or are involved in it.
To change your notification preferences, please click here: http://redmine.yorba.org/my/account
Comment 10
Updated by Jim Nelson 11 months ago
- Category set to network-storage
Comment 11
Updated by Jim Nelson 11 months ago
- Target version changed from 0.14.0 to 0.15.0
Comment 12
Updated by Jim Nelson 9 months ago
This is interesting: https://bugs.launchpad.net/ubuntu/+source/rhythmbox/+bug/955124
Comment 13
Updated by Jim Nelson 6 months ago
-
Target version deleted (
<strike>
_0.15.0_</strike>
)
--- Bug imported by chaz@yorba.org 2013-11-25 21:58 UTC ---
This bug was previously known as bug 5972 at http://redmine.yorba.org/show_bug.cgi?id=5972 Imported an attachment (id=262526)
Unknown version " in product shotwell. Setting version to "!unspecified". Unknown milestone "unknown in product shotwell. Setting to default milestone for this product, "---". Setting qa contact to the default for this product. This bug either had no qa contact or an invalid one. Resolution set on an open status. Dropping resolution
Resolution: RESOLVED INCOMPLETE