Allow apps & components to do openqa testing from their project's Gitlab CI pipelines
We have one set of tests which run against GNOME OS image build from gnome-build-meta 'master' branch. Module maintainers can test changes in their module once they have landed in their project's 'master' branch and been included in a build of GNOME OS. This means it might take a day or two from when a change lands to when it is tested in openQA and issues are discovered.
We want to achieve pre-merge testing using openQA, and we want this both for core apps, core non-apps (e.g. GNOME Shell), and non-core apps (GNOME Circle, etc.)
We need:
- A testsuite that lives in the module's Git tree. Some test helper code will be needed by every module. How do we share that code? Perhaps via a Git submodule?
- A set of Needles for the module's testsuite. Where do these get stored? In the module repo? In https://gitlab.gnome.org/GNOME/openqa-needles?
- A documented way to develop tests locally. This could use ssam_openqa.
- CI scripts to talk to openqa.gnome.org and run isotovideo. This is currently done only in gnome-build-meta's .gitlab-ci.yml. We could extend GNOME citemplates to provide 'openqa' helpers.
- For modules that run on the host, CI scripts to build the module as a systemd sysext, built using the GNOME SDK to ensure compatibility with GNOME OS. (For apps running in Flatpak we can use the flatpak bundle built by flatpak citemplate).
- API token so the project can submit jobs to openQA API - this requires that maintainers have 'Administrator' access to openQA. (see also: #2)
- Gitlab runners with KVM - as of July 2023 all runners should have KVM.
Edited by Sam Thursfield