Juju gives unhelpful error when azure out of resource groups

[Aaron Bentley]

Juju is frequently failing to bootstrap on Azure with messages like "ERROR cmd supercommand.go:448 failed to bootstrap model: creating controller resource group: creating virtual network: creating virtual network "juju-internal-network": autorest:DoErrorUnlessStatusCode PUT https://management.azure.com/subscriptions/2eebf55a-1e02-45d8-a299-02aed8aea00b/resourceGroups/juju-admin-model-848ad671-5eca-4f9a-8fe5-7b7b286e7c90/providers/Microsoft.Network/virtualnetworks/juju-internal-network?api-version=2015-05-01-preview failed with 400 Bad Request"

See here for examples:
http://reports.vapour.ws/releases/issue/5723e7c1749a564b77bc236c

Not clear what the root cause is, but even if it's on the Azure side, we know it doesn't always happen, so there may be a way to mitigate it in Juju.

This is preventing builds from being blessed, so I am marking it critical blocker.

[Andrew Wilkins (axwalk)]

It's hard to tell because the logs disappear when the resource group is deleted by Juju, but I'm guessing this is due to quotas being hit. Your account has many resource groups hanging around, most probably on account of lp:1571687.

[Andrew Wilkins (axwalk)]

Please clean up the leftovers and see if it still happens.

[Aaron Bentley (abentley)]

We just experienced a fresh batch today, and we don't seem to have excessive resource use: 3 virtual machines, 3 cloud services, 4 storage, 4 networks, 1 active directory.

We are already collecting a ton of logs, but if some important ones are missing, please let us know and we can collect them, too.

[Aaron Bentley (abentley)]

Oh, I see there is something actually called resource groups in the new portal, and we do have a ton of them. Fair enough.

[Andrew Wilkins (axwalk)]

Aaron, FYI, there's quite a nice CLI tool for Azure: https://azure.microsoft.com/en-us/documentation/articles/xplat-cli-install/. After installing, I think listing the groups is just a matter of:
    azure config mode arm
    azure login
    azure group list

Then you can "azure group delete <name>" for each one.

[Aaron Bentley (abentley)]

Yes, this does appear to be a resource group problem. So the bug is really that Juju didn't give me an error message I could act on.

[Canonical Juju QA Bot (juju-qa-bot)]

This bug has not been updated in 2 years, so we're marking it Low importance. If you believe this is incorrect, please update the importance.