The file might or might not be inserted to the queue, which meant that
the extra ref created outside the call might never dropped if the file
didn't end up inserted again. Fix this by doing the refcount increase
when actually inserting the file back in the queue.

Reported by Jose M. Arroyo <jose.m.arroyo.se@gmail.com>.