This fixes various reference leaks and correctness issues that could load
to crashes under some scenarios. Much of the async task ownership has been
changed to use the paterns we use else where in Builder.