Multithreaded RAW file processing
Submitted by an unknown user
Assigned to Jim Nelson
Link to original bug (#717131)
Description
---- Reported by shotwell-maint@gnome.bugs 2011-02-06 09:15:00 -0800 ----
Original Redmine bug id: 3157
Original URL: http://redmine.yorba.org/issues/3157
Searchable id: yorba-bug-3157
Original author: Perttu Luukko
Original description:
Hi!
I just started experimenting with Shotwell 0.8.1. When importing my photo library I noticed that the “Processing RAW files†stage takes quite a long time. During this stage Shotwell only uses one of four available cores, so I think a major speed enhancement would be possible by processing the files in parallel.
---- Additional Comments From shotwell-maint@gnome.bugs 2013-05-01 11:39:00 -0700 ----
History
Comment 1
Updated by Adam Dingle almost 3 years ago
- Priority set to High
Our code is actually written so that it can use multiple cores, but we cap the number of cores that are actually used because we've had tricky memory leaks in the past when there was too much work done in parallel. We might be able to improve the current situation, though. Upping to high for more investigation.
Comment 2
Updated by Adam Dingle almost 3 years ago
- Target version set to 0.9
I just spoke to Jim and it turns out that the “processing RAW files†stage is actually fairly independent of the rest of the code, so we may be able to increase the number of cores used here without too much risk. We'd need to do some testing, though, to make sure that the extra cores won't swamp system performance. Under consideration for 0.9.
Comment 3
Updated by Joseph - almost 3 years ago
It's also not necessary to use all the cores. I use dcraw to process my raw files, and I generally reserve 1 processor on a multi core machine, and sometimes as many as 3 on an eight core machine. This generally keeps the system pretty responsive (though I suppose there's also the possibility of running into memory, or io limitations).
Comment 4
Updated by Adam Dingle almost 3 years ago
- Status changed from Open to Review
- Assignee changed from Anonymous to Jim Nelson
Comment 5
Updated by Jim Nelson almost 3 years ago
- Status changed from Review to 5
- Resolution set to fixed
- % Done changed from 0 to 100
I've committed a change that uses all CPUs minus one to generate the mimic files.
r2636
Comment 6
Updated by Charles Lindsay 7 months ago
- Status changed from 5 to Fixed
--- Bug imported by chaz@yorba.org 2013-11-25 21:49 UTC ---
This bug was previously known as bug 3157 at http://redmine.yorba.org/show_bug.cgi?id=3157
Unknown Component Using default product and component set in Parameters Unknown milestone "unknown in product shotwell. Setting to default milestone for this product, "---". Setting qa contact to the default for this product. This bug either had no qa contact or an invalid one.
Version: 0.9
Resolution: RESOLVED FIXED