Speed up CI pipeline proposal
- Complaints about the slow pipeline (cc @svenfoo )
- Less and less C, more an more Rust
- Fail fast, save electricity. For example no need to run anything if
cargo checkfails, no need to do the full
make checkif unit tests fail. Use case is newcomers and project contributors probably won't fiddle with C, but Rust.
- Rust development woes: not having to worry about making both tests green and
clippyhappy at the same time, the pipeline will run them in parallel and complain separately.
- Caching that doesn't slow the builds. Some trial and error showed caching
./targetproduces 400 MB artifacts, for very little speed gain as compiling is reasonably fast. Cache is currently only consisted of cargo crates cache and shared across branches depending on
- Tarball and cross distro can be run manually on request and perhaps scheduled (nightly/weekly) builds. One can still trigger them by clicking play on the separate jobs or an entire step (or per docs in the PR), screenshot below.
The above is all up for discussion.
- didn't mean to rewrite everything nor invalidate any previous work, just personally found the YAML anchors unparsable hence utilized other options where possible
- GitLab does support build matrix but it's under a feature flag which requires an admin to fire up Rails console, hence it was easier to DIY the "Cross Distro"
Tried and gave up on:
- Test reports
- They require converting Tap to JUnit with questionable added value. Demo here.
- The Perl Tap library chokes on custom output, but a Python lib doesn't and it probably doesn't make sense to bloat the docker images with any due to questionable added value.
- Created a small Rust project to consume clippy output (
cargo clippy --message-format=json &> foo.json) and convert it to desired output
- Neither setting
artifacts:reports:codequalitynor copying the config from the example project didn't work
- Screenshot reports are under a feature flag
The above run for 5-6 minutes.
The whole pipeline runs around 20 minutes (with or without cross distro in parallel).
- gather feedback
- put back all the features
- remove debug output
- figure out if C cache makes a difference > it doesn't that much
- check if pages job works > it' does's complicated