excessive memory usage when importing large videos
Submitted by Daniel Gnoutcheff
Link to original bug (#732663)
Description
If I use the 'import from camera' feature to import a large (e.g.
=500MB) video, then shotwell will use a tremendous amount of anonymous memory during the import. The amount of extra memory used seems to be roughly equal to the size of the video being imported.
For example, the attached 'shotwell-1GB-video-import-pmap-log.txt' is a recording of memory usage summaries (from 'pmap -X $(pidof shotwell) | tail -n1') taken every 5 seconds while I imported a ~1 GB video from a Canon PowerShot A570. Notice that the anonymous memory usage gradually increases until it's about 1 GB greater than the pre-import level. Note, too, that my system had to swap stuff out while this was happening. Fortunately, memory usage drops back down to normal levels about 2/3rds of the way though the import.
A similar pattern appears when I import a ~500 MB video, except the memory usage increase is "only" about 500 MB. This problem also occurs when importing from my laptop's SD card reader. However, it does not occur if I just drop the video into ~/Pictures and let the library watcher handle it.
Notably, the gphoto2 command line tool does not suffer from this bug; its memory usage stays flat throughout the transfer.
This was previously reported by Matthijs ten Kate against the Ubuntu package: https://bugs.launchpad.net/ubuntu/+source/shotwell/+bug/912964 In that report, Matthijs discussed both the memory usage and the inadequate progress feedback during large video imports. The progress feedback part was upstreamed as bug 718494; this bug is for the memory usage part.
Resolution: RESOLVED FIXED