simplestreams needs to be updated for Azure Resource Manager

[Andrew Wilkins]

Canonical publishes simplestreams data for Azure images, and also mirrors agents. We need to be able to continue using the former for the old Azure provider, and the latter for both the old and new Azure providers.

For agents, there are two issues:
 - the region names are incompatible between old and new providers. For the old provider, where we would use "Southeast Asia", the new provider uses "southeastasia".
 - the endpoints for some regions in the metadata are wrong: anything that says "https://management.core.windows.net/" should be "https://management.azure.com"

I'll hack around the first issue for now, but it would be great if we could publish with both variations. We need to fix the invalid endpoints though; that's an issue for both the old and new providers.

In addition, we need new image simplestreams data for the new provider. We have image selection built-in to the provider for now, but this is not ideal as it means we have to push code changes if image URNs change significantly.

[Alexis Bruemmer (alexis-bruemmer)]

This needs to be verified once the ARM provider is updated to use simplestreams

[Andrew Wilkins (axwalk)]

I have tested the streams data at http://abentley-streams.s3.amazonaws.com/azure-image-streams, and was able to bootstrap xenial and add a win2012r2 machine.

The xenial machine was started using a beta image. That's because there are beta images in the stream (look for "16.04-beta" in http://abentley-streams.s3.amazonaws.com/azure-image-streams/streams/v1/com.ubuntu.cloud-released-azure.json). I think these should be filtered out. For Ubuntu series, I suggest looking for offers matching specifically:

  YY.MM.N or YY.MM.N-LTS (released stream)
  YY.MM.N-DAILY or YY.MM.N-DAILY-LTS (daily stream)

There's at least one missing region (centralus) for Xenial. It appears to be there for CentOS and Windows images though. Things worked well for me in the "australiasoutheast" region.

[Aaron Bentley (abentley)]

cloudware determines which Ubuntu images to provide for old-azure. We simply mirror that choice for Azure-ARM.

You will find the same images here:
http://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:azure.json

Any corrections to the list of Ubuntu images should be addressed to cloudware.

It is true that centralus is missing for some images, but not for all, and it mirrors cloud-images.

My test streams will lack any changes made to cloud-images since September 24.

[Anastasia (anastasia-macmood)]

Although much work has been done in this area, we will not be addressing this issue fully in 2.1. I am moving it into next milestone.

[Anastasia (anastasia-macmood)]

As this is a tech-debt item, I am lowering its Importance.