SoupCache: add blockfile storage for small resources
@svillar
Submitted by Sergio Assigned to libsoup-maint@gnome.bugs
Link to original bug (#666143)
Description
Both Mozilla and Chromium use some blockfiles to store small resources together. This approach has some advantages:
1- less disk fragmentation. Each small file is stored in a block of the hard disk (block size is commonly 4k) wasting most of its space. For example, in my epy cache I have around 6000 files and ~1300 are smaller than 1k. In that case I'm loosing more than 3K * 1300 = 3900K
2- speed. Reading the contents of small resources from a single mmap'ed file is way faster than opening a bunch of small files. I implemented a small program that reads the contents of ~1300 small files and then the same contents but from a single block file. These are the results (note that I cleared the buffer cache before each test)
-
SSD hard disk: Reading 1369 files lasted: 66109 microseconds Reading a large file lasted: 27557 microseconds (2,4x faster)
-
"normal" laptop hard disk Reading 1369 files lasted: 235907 microseconds Reading a large file lasted: 103494 microseconds (2,28x faster)