README.md 14.6 KB
Newer Older
1
# libpillowfight / pypillowfight
2

3
Really simple C Library containing various image processing algorithms.
Jerome Flesch's avatar
Jerome Flesch committed
4
5
6
7
8
9
10
11
12
13
14
15
It includes Python 3 bindings designed to operate on Pillow images (PIL.Image).

The C library depends only on the libc.
The Python bindings depend only on Pillow.

APIs are designed to be as simple to use as possible. Default values are provided
for every parameters.

Python 2.x is *not* supported.

Available algorithms are listed below.

16
17
## Available algorithms

18
* [Unpaper](https://github.com/Flameeyes/unpaper)'s algorithms
19
20
21
22
23
24
  * Blackfilter
  * Noisefilter
  * Blurfilter
  * Masks
  * Grayfilter
  * Border
25
26
27
* Canny edge detection
* Sobel operator
* Gaussian blur
28
29
* ACE (Automatic Color Equalization ; Parallelized implementation)
* SWT (Stroke Width Transformation)
30
* Compare: Compare two images (grayscale) and makes the pixels that are different
31
  really visible (red).
32
33
* Scan borders: Tries to detect the borders of a page in an image coming from
  a scanner.
34

Jerome Flesch's avatar
Jerome Flesch committed
35
36
37

## Python API

Jerome Flesch's avatar
Jerome Flesch committed
38
The Python API can be compiled, installed and used without installing the C library.
Jerome Flesch's avatar
Jerome Flesch committed
39
40
41

### Installation

42
43
44
45
46
47
48
49
Latest release :

```sh
$ sudo pip3 install pypillowfight
```

Development version :

Jerome Flesch's avatar
Jerome Flesch committed
50
```sh
51
$ git clone https://github.com/openpaperwork/libpillowfight.git
52
$ cd libpillowfight
Jerome Flesch's avatar
Jerome Flesch committed
53

Jerome Flesch's avatar
Jerome Flesch committed
54
# Both C library and Python module
Jerome Flesch's avatar
Jerome Flesch committed
55
56
$ make
$ sudo make install  # will run python3 ./setup.py install + make install (CMake)
Jerome Flesch's avatar
Jerome Flesch committed
57
58
59
60

# Or just the Python bindings
$ make build_py
$ make install_py  # will run only python3 ./setup.py install
Jerome Flesch's avatar
Jerome Flesch committed
61
62
63
64
65
66
67
68
69
70
71
72
```

### Usage

For each algorithm, a function is available. It takes a PIL.Image instance as parameter.
It may take other optionnal parameters. The return value is another PIL.Image instance.

Example:

```py
import pillowfight

73
input_img = PIL.Image.open("tests/data/brightness_problem.png")
Jerome Flesch's avatar
Jerome Flesch committed
74
75
76
77
78
79
output_img = pillowfight.ace(input_img)
```

### Tests

```sh
80
81
make check  # will check style
make test  # will run the tests (will require tox)
Jerome Flesch's avatar
Jerome Flesch committed
82
83
```

84
85
86
87
Test reference images are made on amd64. They should match also on i386.
On other architectures however, due to slight differences regarding floating
point numbers, results may vary slightly and tests may not pass.

Jerome Flesch's avatar
Jerome Flesch committed
88

Jerome Flesch's avatar
Jerome Flesch committed
89
## C library
Jerome Flesch's avatar
Jerome Flesch committed
90
91
92
93

### Installation

```sh
Jerome Flesch's avatar
Jerome Flesch committed
94
95
96
# C library only (will use CMake)
$ make build_c
$ sudo make install_c
Jerome Flesch's avatar
Jerome Flesch committed
97
98
99
100
```

### Usage

101
102
#### C code

Jerome Flesch's avatar
Jerome Flesch committed
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
For each algorithm, a function is available. It takes a ```struct pf_bitmap```
as input. As output, it fills in another ```struct pf_bitmap```.

```struct pf_bitmap``` is a really simple structure:

```C
struct pf_bitmap {
	struct {
		int x;
		int y;
	} size;
	union pf_pixel *pixels;
};
```

```(struct pf_bitmap).size.x``` is the width of the image.

```(struct pf_bitmap).size.y``` is the height of the image.

```union pf_pixel``` are basically 32 bits integers, defined in a manner convenient
Jerome Flesch's avatar
Jerome Flesch committed
123
to retrieve each color independantly (RGB). Each color is on one byte. 4th byte is
Jerome Flesch's avatar
Jerome Flesch committed
124
125
126
unused (no alpha channel taken into account).

```(struct pf_bitmap).pixels``` must points to a memory area containing the image.
Jerome Flesch's avatar
Jerome Flesch committed
127
The image must contains ```x * y * union pf_pixel```.
Jerome Flesch's avatar
Jerome Flesch committed
128
129


130
131
132
133
134
135
136
#### Compilation with GCC

```
$ gcc -Wall -Werror -lpillowfight -o test test.c
```


Jerome Flesch's avatar
Jerome Flesch committed
137
## Note regarding Unpaper's algorithms
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158

Many algorithms in this library are re-implementations of algorithms used
by [Unpaper](https://github.com/Flameeyes/unpaper). To make the API simpler
to use (.. and implement), a lot of settings have been hard-coded.

Unpaper applies them in the following order:

* Blackfilter
* Noisefilter
* Blurfilter
* Masks
* Grayfilter
* Border

I would advise applying automatic color equalization (ACE) first.

A basic documentation for some of the algorithms can be found in
[Unpaper's documentation](https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md).

| Input | Output |
| ----- | ------ |
159
160
| [Black border problem](tests/data/black_border_problem.png) | [ACE + Unpapered](tests/data/black_border_problem_all.png) |
| [Brightness problem](tests/data/brightness_problem.png) | [ACE + Unpapered](tests/data/brightness_problem_all.png) |
161

Jerome Flesch's avatar
Jerome Flesch committed
162
163
164
165
## Available algorithms

### Automatic Color Equalization (ACE)

Jerome Flesch's avatar
Jerome Flesch committed
166
167
| Input | Output |
| ----- | ------ |
168
| [Brightness problem](tests/data/brightness_problem.png) | [Corrected](tests/data/brightness_problem_ace.png) |
Jerome Flesch's avatar
Jerome Flesch committed
169

Jerome Flesch's avatar
Jerome Flesch committed
170
171
172
This algorithm is quite slow (~40s for one big image with one thread
on my machine). So this version is parallelized (down to ~15s on a 4
cores computer).
Jerome Flesch's avatar
Jerome Flesch committed
173

Jerome Flesch's avatar
Jerome Flesch committed
174

Jerome Flesch's avatar
Jerome Flesch committed
175
176
177
178
179
180
181
182
183
184
185
186
#### Python API

```py
out_img = pillowfight.ace(img_in,
	slope=10,
	limit=1000,
	samples=100,
	seed=None)
```

Use as many threads as there are cores on the computer (up to 32).

187
188
189
190
191
This algorithm uses random number. If you need consistent results
(for unit tests for instance), you can specify a seed for the
random number generator. Otherwise, time.time() will be used.


Jerome Flesch's avatar
Jerome Flesch committed
192
193
194
195
196
197
198
199
200
201
202
203
#### C API

```C
#define PF_DEFAULT_ACE_SLOPE 10
#define PF_DEFAULT_ACE_LIMIT 1000
#define PF_DEFAULT_ACE_NB_SAMPLES 100
#define PF_DEFAULT_ACE_NB_THREADS 2
extern void pf_ace(const struct pf_bitmap *in, struct pf_bitmap *out,
		int nb_samples, double slope, double limit,
		int nb_threads);
```

204
205
206
This function uses random numbers coming (```rand()```).
You *should* call ```srand()``` before calling this function.

Jerome Flesch's avatar
Jerome Flesch committed
207
208
209
210
211
212
213

#### Sources

* "A new algorithm for unsupervised global and local color correction." - A. Rizzi, C. Gatta and D. Marini
* http://argmax.jp/index.php?colorcorrect


214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
### Scan border

This algorithm tries to find page borders in a scanned image. It is designed
to operate on images coming from a flatbed scanner or a scanner with an
automatic document feeder.

This algorithms looks for horizontal and vertical lines, and return the
smallest rectangle that includes all those lines. To get the lines, it runs
the Sobel operator on the input image and only keep the points with
an angle of [0°, 90°, 180°, 270°] ±5°.

This algorithm does not always work:
- It's quite sensible to noise: dust, hair, etc may easily be counted
  erroneously as lines.
- Some scanners or drivers (most of Brother scanners for instance) "clean up"
  the image before returning it. Unfortunately they often remove most of the
  page borders in the process.

Jerome Flesch's avatar
0.3.0    
Jerome Flesch committed
232
233
Still, this algorithm can help users of GUI applications by pre-selecting a
cropping area.
234
235
236
237


| Input | Output |
| ----- | ------ |
238
239
240
| [brother_mfc7360n](tests/data/brother_mfc7360.png) | (56, 8, 1637, 2275) |
| [epson_xp425](tests/data/epson_xp425.png) | (4, 5, 2484, 3498) |
| [brother_ds620](tests/data/brother_ds620.png) | (3, 3, 2507, 3527) |
241
242
243
244

#### Python API

```py
Jerome Flesch's avatar
Jerome Flesch committed
245
frame = pillowfight.find_scan_borders(img_in)
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
```


#### C API

```C
struct pf_point {
	int x;
	int y;
};

struct pf_rectangle {
	struct pf_point a;
	struct pf_point b;
};

Jerome Flesch's avatar
Jerome Flesch committed
262
struct pf_rectangle pf_find_scan_borders(const struct pf_bitmap *img_in);
263
264
265
```


Jerome Flesch's avatar
Jerome Flesch committed
266
267
268
269
### Canny's edge detection

| Input | Output |
| ----- | ------ |
270
| [Crappy background](tests/data/crappy_background.png) | [Canny output](tests/data/crappy_background_canny.png) |
Jerome Flesch's avatar
Jerome Flesch committed
271

Jerome Flesch's avatar
Jerome Flesch committed
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292

#### Python API

```py
img_out = pillowfight.canny(img_in)
```


#### C API

```C
extern void pf_canny(const struct pf_bitmap *in, struct pf_bitmap *out);
```


#### Sources

* "A computational Approach to Edge Detection" - John Canny
* https://en.wikipedia.org/wiki/Canny_edge_detector


Jerome Flesch's avatar
Jerome Flesch committed
293
### Compare
294
295
296
297

Simple algorithm showing the difference between two images.
Note that it converts the images to grayscale first.

298
It accepts a parameter 'tolerance': For each pixel, the difference with
Jerome Flesch's avatar
Jerome Flesch committed
299
300
301
the corresponding pixel from the other image is computed. If the
difference is between 0 and 'tolerance', it is ignored (pixels
are considered equal).
302

303
304
| Input | Input2 | Output |
| ----- | ------ | ------ |
305
| [Black border problem](tests/data/black_border_problem.png) | [Black border problem + blackfilter](tests/data/black_border_problem_blackfilter.png) | [Diff](tests/data/black_border_problem_diff.png) |
306
307
308
309

#### Python API

```py
310
(nb_diff, out_img) = pillowfight.compare(img_in, img_in2, tolerance=10)
311
312
```

313

314
315
316
#### C API

```C
Jerome Flesch's avatar
Jerome Flesch committed
317
extern int pf_compare(const struct pf_bitmap *in, const struct pf_bitmap *in2,
318
		struct pf_bitmap *out, int tolerance);
319
320
```

321
322
Returns the number of pixels that are different between both images.

323

Jerome Flesch's avatar
Jerome Flesch committed
324
325
### Gaussian

Jerome Flesch's avatar
Jerome Flesch committed
326
327
| Input | Output |
| ----- | ------ |
328
| [Crappy background](tests/data/crappy_background.png) | [Gaussed](tests/data/crappy_background_gaussian.png) |
Jerome Flesch's avatar
Jerome Flesch committed
329

330
331
332
333
334
335
One of the parameters is ```sigma```. If it is equals to 0.0, it will be computed automatically
using the following formula (same as OpenCV):

```C
sigma = 0.3 * ((nb_stddev - 1) * 0.5 - 1) + 0.8;
```
Jerome Flesch's avatar
Jerome Flesch committed
336

Jerome Flesch's avatar
Jerome Flesch committed
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
#### Python API

```py
img_out = pillowfight.gaussian(img_in, sigma=2.0, nb_stddev=5)
```


#### C API

```
extern void pf_gaussian(const struct pf_bitmap *in, struct pf_bitmap *out,
	double sigma, int nb_stddev);
```


#### Sources

* https://en.wikipedia.org/wiki/Gaussian_blur
* https://en.wikipedia.org/wiki/Gaussian_function


### Sobel operator

Jerome Flesch's avatar
Jerome Flesch committed
360
361
| Input | Output |
| ----- | ------ |
362
| [Crappy background](tests/data/crappy_background.png) | [Sobel](tests/data/crappy_background_sobel.png) |
Jerome Flesch's avatar
Jerome Flesch committed
363
364


Jerome Flesch's avatar
Jerome Flesch committed
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
#### Python API

```py
img_out = pillowfight.sobel(img_in)
```


#### C API

```C
extern void pf_sobel(const struct pf_bitmap *in_img, struct pf_bitmap *out_img);
```


#### Sources

* https://www.researchgate.net/publication/239398674_An_Isotropic_3_3_Image_Gradient_Operator
* https://en.wikipedia.org/wiki/Sobel_operator


### Stroke Width Transformation

387
This algorithm extracts text from natural scenes images.
388
389
390
391
392
393
394
395
396
397
398
399

To find text, it looks for strokes. Note that it doesn't appear to work well on
scanned documents because strokes are too small.

This implementation can provide the output in 3 different ways:

* Black & White : Detected text is black. Background is white.
* Grayscale : Detected text is gray. Its exact color is proportional to the stroke width detected.
* Original boxes : The rectangle around the detected is copied as is in the output image. Rest of the image is white.

(following examples are with original boxes)

Jerome Flesch's avatar
Jerome Flesch committed
400
401
| Input | Output |
| ----- | ------ |
402
403
404
405
| [Black border problen](tests/data/black_border_problem.png) | [SWT (SWT_OUTPUT_ORIGINAL_BOXES)](tests/data/black_border_problem_swt.png) |
| [Crappy background](tests/data/crappy_background.png) | [SWT (SWT_OUTPUT_ORIGINAL_BOXES)](tests/data/crappy_background_swt.png) |
| [Black border problen](tests/data/black_border_problem.png) | [SWT (SWT_OUTPUT_BW_TEXT)](tests/data/black_border_problem_swt.png) |
| [Crappy background](tests/data/crappy_background.png) | [SWT (SWT_OUTPUT_BW_TEXT)](tests/data/crappy_background_swt.png) |
Jerome Flesch's avatar
Jerome Flesch committed
406
407


Jerome Flesch's avatar
Jerome Flesch committed
408
409
410
#### Python API

```py
411
412
413
414
# SWT_OUTPUT_BW_TEXT = 0  # default
# SWT_OUTPUT_GRAYSCALE_TEXT = 1
# SWT_OUTPUT_ORIGINAL_BOXES = 2

415
img_out = pillowfight.swt(img_in, output_type=pillowfight.SWT_OUTPUT_ORIGINAL_BOXES)
Jerome Flesch's avatar
Jerome Flesch committed
416
417
418
419
420
421
```


#### C API

```C
422
423
424
425
426
427
428
429
430
431
enum pf_swt_output
{
	PF_SWT_OUTPUT_BW_TEXT = 0,
	PF_SWT_OUTPUT_GRAYSCALE_TEXT,
	PF_SWT_OUTPUT_ORIGINAL_BOXES,
};
#define PF_DEFAULT_SWT_OUTPUT PF_SWT_OUTPUT_BW_TEXT

extern void pf_swt(const struct pf_bitmap *in_img, struct pf_bitmap *out_img,
		enum pf_swt_output output_type);
Jerome Flesch's avatar
Jerome Flesch committed
432
433
434
435
436
437
438
439
440
441
442
```


#### Sources

* ["Detecting Text in Natural Scenes with Stroke Width Transform"](http://cmp.felk.cvut.cz/~cernyad2/TextCaptchaPdf/Detecting%20Text%20in%20Natural%20Scenes%20with%20Stroke%20Width%20Transform.pdf) - Boris Epshtein, Eyal Ofek, Yonatan Wexler
* https://github.com/aperrau/DetectText


### Unpaper's Blackfilter

443
444
| Input | Output | Diff |
| ----- | ------ | ---- |
445
| [Black border problem](tests/data/black_border_problem.png) | [Filtered](tests/data/black_border_problem_blackfilter.png) | [Diff](tests/data/black_border_problem_blackfilter_diff.png) |
Jerome Flesch's avatar
Jerome Flesch committed
446
447


Jerome Flesch's avatar
Jerome Flesch committed
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
#### Python API

```py
img_out = pillowfight.unpaper_blackfilter(img_in)
```


#### C API

```C
extern void pf_unpaper_blackfilter(const struct pf_bitmap *in, struct pf_bitmap *out);
```


#### Sources

* https://github.com/Flameeyes/unpaper
* https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md


### Unpaper's Blurfilter

470
471
| Input | Output | Diff |
| ----- | ------ | ---- |
472
| [Black border problem](tests/data/black_border_problem.png) | [Filtered](tests/data/black_border_problem_blurfilter.png) | [Diff](tests/data/black_border_problem_blurfilter_diff.png) |
Jerome Flesch's avatar
Jerome Flesch committed
473
474


Jerome Flesch's avatar
Jerome Flesch committed
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
#### Python API

```py
img_out = pillowfight.unpaper_blurfilter(img_in)
```


#### C API

```C
extern void pf_unpaper_blurfilter(const struct pf_bitmap *in, struct pf_bitmap *out);
```


#### Sources

* https://github.com/Flameeyes/unpaper
* https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md


### Unpaper's Border

497
498
| Input | Output | Diff |
| ----- | ------ | ---- |
499
| [Black border problem 3](tests/data/black_border_problem3.png) | [Border](tests/data/black_border_problem3_border.png) | [Diff](tests/data/black_border_problem3_border_diff.png) |
Jerome Flesch's avatar
Jerome Flesch committed
500
501


Jerome Flesch's avatar
Jerome Flesch committed
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
#### Python API

```py
img_out = pillowfight.unpaper_border(img_in)
```


#### C API

```C
extern void pf_unpaper_border(const struct pf_bitmap *in, struct pf_bitmap *out);
```


#### Sources

* https://github.com/Flameeyes/unpaper
* https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md


### Unpaper's Grayfilter

524
525
| Input | Output | Diff |
| ----- | ------ | ---- |
526
| [Black border problem 3](tests/data/black_border_problem.png) | [Filterd](tests/data/black_border_problem_grayfilter.png) | [Diff](tests/data/black_border_problem_grayfilter_diff.png) |
Jerome Flesch's avatar
Jerome Flesch committed
527
528


Jerome Flesch's avatar
Jerome Flesch committed
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
#### Python API

```py
img_out = pillowfight.unpaper_grayfilter(img_in)
```


#### C API

```C
extern void pf_unpaper_grayfilter(const struct pf_bitmap *in, struct pf_bitmap *out);
```


#### Sources

* https://github.com/Flameeyes/unpaper
* https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md


### Unpaper's Masks

551
552
| Input | Output | Diff |
| ----- | ------ | ---- |
553
| [Black border problem 2](tests/data/black_border_problem2.png) | [Masks](tests/data/black_border_problem2_masks.png) | [Diff](tests/data/black_border_problem2_masks_diff.png) |
Jerome Flesch's avatar
Jerome Flesch committed
554
555


Jerome Flesch's avatar
Jerome Flesch committed
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
#### Python API

```py
img_out = pillowfight.unpaper_masks(img_in)
```


#### C API

```C
extern void pf_unpaper_masks(const struct pf_bitmap *in, struct pf_bitmap *out);
```


#### Sources

* https://github.com/Flameeyes/unpaper
* https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md


### Unpaper's Noisefilter

578
579
| Input | Output | Diff |
| ----- | ------ | ---- |
580
| [Black border problem](tests/data/black_border_problem.png) | [Filtered](tests/data/black_border_problem_noisefilter.png) | [Diff](tests/data/black_border_problem_noisefilter_diff.png) |
Jerome Flesch's avatar
Jerome Flesch committed
581
582


Jerome Flesch's avatar
Jerome Flesch committed
583
584
585
586
587
588
589
590
591
592
593
594
595
596
#### Python API

```py
img_out = pillowfight.unpaper_noisefilter(img_in)
```


#### C API

```C
extern void pf_unpaper_noisefilter(const struct pf_bitmap *in, struct pf_bitmap *out);
```


Jerome Flesch's avatar
Jerome Flesch committed
597
598
## Contact

599
* [Forum](https://forum.openpaper.work/)
600
* [Bug tracker](https://github.com/openpaperwork/libpillowfight/issues/)
Jerome Flesch's avatar
Jerome Flesch committed
601
602


Jerome Flesch's avatar
Jerome Flesch committed
603
604
605
606
#### Sources

* https://github.com/Flameeyes/unpaper
* https://github.com/Flameeyes/unpaper/blob/master/doc/basic-concepts.md