memory exhaustion on video upload
Submitted by an unknown user
Assigned to Jim Nelson
Link to original bug (#717278)
Description
---- Reported by shotwell-maint@gnome.bugs 2010-12-18 13:01:00 -0800 ----
Original Redmine bug id: 2991
Original URL: http://redmine.yorba.org/issues/2991
Searchable id: yorba-bug-2991
Original author: Jani Monoses
Original description:
Maybe issue 2990 is due to uploadingtmany videos totaling a couple of Gb which may be kept in RAM until the upload finishes.
A second attempt at uploading ~10 videos brought the computer to a halt after about 20 minutes, after uploading about half of them.
top showed about 1.8G virt 1.5G RSS memory being used by Shotwell before the system became unusable.
---- Additional Comments From shotwell-maint@gnome.bugs 2013-05-01 11:39:00 -0700 ----
History
Comment 1
Updated by Adam Dingle almost 3 years ago
-
Priority deleted (
<strike>
__</strike>
)
Comment 2
Updated by Jim Nelson almost 3 years ago
- Status changed from Open to Review
- Assignee changed from Anonymous to Jim Nelson
See note at #2990 on why I think these two are related.
Comment 3
Updated by Adam Dingle almost 3 years ago
In related news, I'm seeing a memory leak related to video publishing. Steps to reproduce:
Select a good-sized video and choose Publish.
Choose a publishing service. I'm using Facebook; not sure whether this matters.
Begin uploading the video.
Cancel the publishing operation.
If you perform the steps above repeatedly, you'll see Shotwell's memory usage climb without bound in System Monitor. I was using a 30-megabyte video, and here are the memory values I observed at each step:| iteration | Memory (Mb) | Virtual Memory (Mb) |
0
20
508
1
82
573
2
112
604
3
143
634
4
173
665
5
204
695
Comment 4
Updated by Jim Nelson almost 3 years ago
- Status changed from Review to 5
- Resolution set to fixed
- % Done changed from 0 to 100
It does appear that this and #2990 are related, and I can now see why. Although each video is uploaded one at a time, and loaded into memory one at a time, the way the transaction objects were being processed the completed objects would not be freed until they all were completed, so memory usage grows and grows until the entire operation is finished.
However, that doesn't explain the persistent memory leak Adam was seeing. That was due to both a reference cycle and a bad Vala binding that ensured that Soup messages (the very objects holding the video file in memory) were never released. In essence, the entire publishing data set was remaining in memory after the operation was completed or canceled.
I think this patch nails it down. I've been uploading to YouTube and memory never grows above 80M on my small set of files. A thorough test of the publishing subsystem -- photos and videos -- would be wise at this point.
Comment 5
Updated by Adam Dingle almost 3 years ago
Wow – congratulations on tracking this down. Great work!
Comment 6
Updated by Charles Lindsay 7 months ago
- Status changed from 5 to Fixed
--- Bug imported by chaz@yorba.org 2013-11-25 21:50 UTC ---
This bug was previously known as bug 2991 at http://redmine.yorba.org/show_bug.cgi?id=2991
Unknown Component Using default product and component set in Parameters Unknown version " in product shotwell. Setting version to "!unspecified". Unknown milestone "unknown in product shotwell. Setting to default milestone for this product, "---". Setting qa contact to the default for this product. This bug either had no qa contact or an invalid one.
Resolution: RESOLVED FIXED