meson test: Discover/list individual tests by file
This is very much a take-it-or-leave-it MR. There are certainly arguments to be made against it, because it decouples test discovery from pytest
itself. But, the experience strikes me as somewhat more comfortable, particularly if tests start failing. (At least you'd get a vague idea which ones, right up front.)
Instead of just reporting one monolithic 'pygobject_test_suite' when running unit tests in meson, use a Python script to find all of the unit test files in the tests/
dir, and create a separate meson unit test for each file.
Tests are discovered using the tests/list_tests.py
script, which (non-recursively) finds all files in the same directory which are named like either test_*.py
or *_test.py
(although it doesn't use glob.glob
).
On my system, this adds a negligible amount of runtime to the test suite — ~11.5-12.5s vs. ~10-11.5s. (Measured using time
on the entire meson test -C _build
command execution, with a hot builddir cache and no configure/compile actions required.)
Before
$ meson test -C _build
ninja: Entering directory `.../_build'
ninja: no work to do.
1/1 pygobject-test-suite OK 11.24s
Ok: 1
Expected Fail: 0
Fail: 0
Unexpected Pass: 0
Skipped: 0
Timeout: 0
Full log written to .../_build/meson-logs/testlog.txt
After
$ meson test -C _build
ninja: Entering directory `.../_build'
ninja: no work to do.
1/41 atoms OK 0.85s
2/41 docstring OK 0.86s
3/41 error OK 0.87s
4/41 cairo OK 0.93s
5/41 fields OK 0.88s
6/41 gdbus OK 0.86s
7/41 fundamental OK 0.97s
8/41 everything OK 1.45s
9/41 generictreemodel OK 1.04s
10/41 gio OK 0.99s
11/41 glib OK 0.99s
12/41 gi OK 1.97s
13/41 gtk template OK 0.95s
14/41 gobject OK 1.11s
15/41 gtype OK 0.88s
16/41 import machinery OK 0.83s
17/41 interface OK 0.83s
18/41 internal api OK 0.88s
19/41 iochannel OK 0.89s
20/41 mainloop OK 0.94s
21/41 option OK 0.84s
22/41 ossig OK 0.98s
23/41 object marshaling OK 1.59s
24/41 overrides gdk OK 1.02s
25/41 overrides gdkpixbuf OK 1.15s
26/41 overrides glib OK 0.91s
27/41 overrides gio OK 1.10s
28/41 overrides gobject OK 0.88s
29/41 overrides pango OK 0.83s
30/41 properties OK 1.16s
31/41 pycapi OK 0.92s
32/41 repository OK 0.90s
33/41 resulttuple OK 0.85s
34/41 overrides gtk OK 3.22s
35/41 pygtkcompat OK 2.41s
36/41 signal OK 1.42s
37/41 source OK 1.55s
38/41 subprocess OK 1.17s
39/41 thread OK 0.88s
40/41 typeclass OK 0.88s
41/41 unknown OK 0.79s
Ok: 41
Expected Fail: 0
Fail: 0
Unexpected Pass: 0
Skipped: 0
Timeout: 0
Full log written to .../_build/meson-logs/testlog.txt