Skip to content

meson test: Discover/list individual tests by file

This is very much a take-it-or-leave-it MR. There are certainly arguments to be made against it, because it decouples test discovery from pytest itself. But, the experience strikes me as somewhat more comfortable, particularly if tests start failing. (At least you'd get a vague idea which ones, right up front.)

Instead of just reporting one monolithic 'pygobject_test_suite' when running unit tests in meson, use a Python script to find all of the unit test files in the tests/ dir, and create a separate meson unit test for each file.

Tests are discovered using the tests/list_tests.py script, which (non-recursively) finds all files in the same directory which are named like either test_*.py or *_test.py (although it doesn't use glob.glob).

On my system, this adds a negligible amount of runtime to the test suite — ~11.5-12.5s vs. ~10-11.5s. (Measured using time on the entire meson test -C _build command execution, with a hot builddir cache and no configure/compile actions required.)

Before

$ meson test -C _build
ninja: Entering directory `.../_build'
ninja: no work to do.
1/1 pygobject-test-suite        OK             11.24s

Ok:                 1   
Expected Fail:      0   
Fail:               0   
Unexpected Pass:    0   
Skipped:            0   
Timeout:            0   

Full log written to .../_build/meson-logs/testlog.txt

After

$ meson test -C _build
ninja: Entering directory `.../_build'
ninja: no work to do.
 1/41 atoms                       OK              0.85s
 2/41 docstring                   OK              0.86s
 3/41 error                       OK              0.87s
 4/41 cairo                       OK              0.93s
 5/41 fields                      OK              0.88s
 6/41 gdbus                       OK              0.86s
 7/41 fundamental                 OK              0.97s
 8/41 everything                  OK              1.45s
 9/41 generictreemodel            OK              1.04s
10/41 gio                         OK              0.99s
11/41 glib                        OK              0.99s
12/41 gi                          OK              1.97s
13/41 gtk template                OK              0.95s
14/41 gobject                     OK              1.11s
15/41 gtype                       OK              0.88s
16/41 import machinery            OK              0.83s
17/41 interface                   OK              0.83s
18/41 internal api                OK              0.88s
19/41 iochannel                   OK              0.89s
20/41 mainloop                    OK              0.94s
21/41 option                      OK              0.84s
22/41 ossig                       OK              0.98s
23/41 object marshaling           OK              1.59s
24/41 overrides gdk               OK              1.02s
25/41 overrides gdkpixbuf         OK              1.15s
26/41 overrides glib              OK              0.91s
27/41 overrides gio               OK              1.10s
28/41 overrides gobject           OK              0.88s
29/41 overrides pango             OK              0.83s
30/41 properties                  OK              1.16s
31/41 pycapi                      OK              0.92s
32/41 repository                  OK              0.90s
33/41 resulttuple                 OK              0.85s
34/41 overrides gtk               OK              3.22s
35/41 pygtkcompat                 OK              2.41s
36/41 signal                      OK              1.42s
37/41 source                      OK              1.55s
38/41 subprocess                  OK              1.17s
39/41 thread                      OK              0.88s
40/41 typeclass                   OK              0.88s
41/41 unknown                     OK              0.79s

Ok:                 41  
Expected Fail:      0   
Fail:               0   
Unexpected Pass:    0   
Skipped:            0   
Timeout:            0   

Full log written to .../_build/meson-logs/testlog.txt

Merge request reports