gitlab-ci: don't perform rebuilds at test phase
Just use artifacts as it's what gitlab-ci provides.
While this might not be strictly needed to save time by the caching already in place, it's just preferable to have a separation of concerns, where build phase builds only, and test phase tests only.
Caching time could be even reduced in case, another thing that could be done is to check for artifacts availability and if the build dir isn't there to launch building again. But I think this should not really be ever needed if there are no issues at CI servers level.