Use prebuilt container on arm64

Use a custom container so we don't run into timeouts and use less space during build.

I'm posting this here since it helped me a lot to get a working build for arm64. No urgency to get it merged.