... | ... | @@ -10,23 +10,29 @@ OpenQA has some useful online documentation. Be sure to look at the |
|
|
|
|
|
Deployment of openqa.gnome.org is documented [here](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/wikis/openqa/Deployment).
|
|
|
|
|
|
## Running the OpenQA tests
|
|
|
## Checking test results
|
|
|
|
|
|
The OpenQA tests run as part of Gitlab CI and can be started manually as follows.
|
|
|
1. Look at the latest 'master' pipelines for gnome-build-meta: https://gitlab.gnome.org/GNOME/gnome-build-meta/-/pipelines?page=1&scope=all&ref=master
|
|
|
|
|
|
On an open merge request:
|
|
|
2. Open the 'test-s3-image' job:
|
|
|
|
|
|
1. Wait for 'track' and 'build-x86_64' jobs to complete
|
|
|
1. Start 'iso-installer-x86_64' job.
|
|
|
1. Start 'test-iso-installer-x86_64' job.
|
|
|
1. Visit https://openqa.gnome.org/ to see progress (a link will be printed in Gitlab CI log)
|
|
|
![image](uploads/81de9378eec4a4857ff5c5cbf8dae83b/image.png)
|
|
|
|
|
|
To run the tests against 'master' or a branch with no open merge request,
|
|
|
first [start a pipeline](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/pipelines)
|
|
|
and then follow the steps above.
|
|
|
3. Click the link in the log to open the OpenQA web UI:
|
|
|
|
|
|
We would like to run the tests for every MR, which requires us to first [speed up the iso-installer job](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues/396).
|
|
|
![image](uploads/dcca54491c5993122a8fbd3539fe02f6/image.png)
|
|
|
|
|
|
## Running the OpenQA tests manually
|
|
|
|
|
|
There are currently two ways to run the tests in Gitlab CI.
|
|
|
|
|
|
One, is to trigger a [gnome-build-meta pipeline](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/pipelines) against the 'master' branch or a release tag. This builds an image from scratch which is [slow](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues/396).
|
|
|
|
|
|
The other is to trigger a pipeline in the [openqa-tests](https://gitlab.gnome.org/GNOME/openqa-tests) repo. This will run against the current [os.gnome.org](https://os.gnome.org/) image, which is fast. This is very useful when developing new tests.
|
|
|
|
|
|
It's not currently possible to run the openQA tests against regular merge requests to gnome-build-meta. See https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues/601 to see what is needed.
|
|
|
|
|
|
See below for instructions on running the test suite on your local machine.
|
|
|
|
|
|
## When tests fail
|
|
|
|
... | ... | |