Integration testing the engine
Geary really needs to run automated integration tests against actual mail servers so that changes to the engine can be made with confidence. There should probably be a generic test driver for both IMAP and SMTP that accepts a ServiceInformation config and runs a series of tests against that (connect, starttls, login, list folders, send mail, etc).
That can be built on by adding a series of drivers that each launches a container/jail/vm/etc with a specific mail server daemons, then runs the generic test driver against that. These could be run by anyone with the container tech installed, and possibly also by GitLab' CI process.
To handle testing against cloud providers, we don't want to commit login and passwords to the repo. So GitLab CI variables could be set for automated testing there, and people could create a config file with login details for their own private test accounts in their working dir.
The work involved is probably something like:
- Refactor Geary.Imap.ClientSesson and Geary.Smtp.ClientSession to provide a uniform API (useful for writing the test driver, but also would allow the client's and engine's account validation to be cleaned up substantially)
- Implement abstract test drivers for IMAP and for SMTP that runs a bunch of tests against a ServiceInformation
- Implement a generic test driver that reads ServiceInformation config from a file and runs that
- Create some Docker images for target daemons (dovecot, cyrus, courier, uw-imap?, postfix, exim, sendmail)
- Implement an abstract test driver for launching a container and running the appropriate test driver against that
- Implement a specific test driver for each of the containers, hook them up into the build
- Implement specific test drivers for each cloud provider (gmail, outlook, y! icloud, fastmail, mailfence, etc) and handle passing credentials to them somehow
- Get it working with GitLab CI