New units fail provisioning; juju 2.5.x switches to the Standard SKU (Azure provider)

Bug #1824465 reported by Haw Loeung
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
Yang Kelvin Liu
2.5
Fix Released
High
Tim McNamara
2.6
Fix Released
High
Tim McNamara

Bug Description

Hi,

We recently upgraded one of our cloud archive mirrors in Azure to Juju 2.5.4 (from 2.4.7). Unfortunately, this now breaks provisioning of new units because the load balancer "SKU" Juju provisions with has changed from "Basic" to "Standard".

The Azure portal for instances failing provisioning shows the following:

| Additional error information is available for this virtual machine:
| GENERAL
| Provisioning state Provisioning failed. Different basic sku and standard sku load balancer or public Ip resources in availability set is not allowed /subscriptions/bb7e37e8-bc53-403a-a4a9-4dd507d7813b/resourceGroups/juju-archive-mirror-azure-central-us-model-417ca9a1-d880-4c09-885a-8a0e6ee57ee3/providers/Microsoft.Network/publicIPAddresses/machine-0-public-ip and /subscriptions/bb7e37e8-bc53-403a-a4a9-4dd507d7813b/resourceGroups/juju-archive-mirror-azure-central-us-model-417ca9a1-d880-4c09-885a-8a0e6ee57ee3/providers/Microsoft.Network/publicIPAddresses/machine-6-public-ip.. DifferentSkuLoadBalancersAndPublicIPAddressNotAllowed
| Provisioning state error code ProvisioningState/failed/DifferentSkuLoadBalancersAndPublicIPAddressNotAllowed
| Guest agent Not Ready. VM status blob is found but not yet populated.
| DISKS
| machine-6 Provisioning succeeded.

For these environments, they were deployed pre-Juju 2.5.x and therefore using the "basic SKU".

It might be https://github.com/juju/juju/pull/9499 and commit '9cd72eb26a07e4f662caa5d9f89cae4fedabbfb7' switching to "standard".

I think we should detect what was previously used and provision units using matching SKUs. I think we should also provide the ability to override the SKU for those wishing to use either "basic" or "standard".

FWIW, standard was originally requested in LP: #1799238 because it seems to be the one that provides network metrics.

Haw Loeung (hloeung)
summary: - juju 2.5.x switches to the Standard SKU for Azure
+ New units fail provisioning; juju 2.5.x switches to the Standard SKU
+ (Azure provider)
description: updated
Changed in juju:
status: New → Triaged
importance: Undecided → High
milestone: none → 2.6-beta1
Changed in juju:
milestone: 2.6-beta1 → 2.6-beta2
Changed in juju:
milestone: 2.6-beta2 → 2.6-rc1
Changed in juju:
milestone: 2.6-rc1 → 2.6-rc2
Changed in juju:
milestone: 2.6-rc2 → 2.6.1
Changed in juju:
milestone: 2.6.1 → 2.6.2
Changed in juju:
milestone: 2.6.2 → 2.6.3
Changed in juju:
milestone: 2.6.3 → 2.6.4
Revision history for this message
Nick Moffitt (nick-moffitt) wrote :

Is there any workaround you can recommend here? We need to scale out on azure for production services.

Revision history for this message
Joel Sing (jsing) wrote :

This is having a production impact since units cannot be added to any Azure model created with an older Juju release.

Revision history for this message
Ian Booth (wallyworld) wrote :

This is top of the queue to address. Sadly there doesn't appear to be a work around as far as I know. We'll look to get a fix out asap.

Changed in juju:
assignee: nobody → Tim McNamara (tim-clicks)
status: Triaged → In Progress
tags: added: azure-provider
Changed in juju:
milestone: 2.6.4 → 2.6.5
Revision history for this message
Tim McNamara (tim-clicks) wrote :

A fix[0] for this issue has been merged into the Juju 2.5 branch. This will be released as part of 2.5.8.

[0]: https://github.com/juju/juju/pull/10355

Changed in juju:
milestone: 2.6.5 → 2.5.8
status: In Progress → Fix Committed
Revision history for this message
Joel Sing (jsing) wrote :

Thanks for the fix - I presume this will also be addressed in the 2.6.x series?

Revision history for this message
Anastasia (anastasia-macmood) wrote :

@Joel Sing (jsing),

The fix will be forward-ported to 2.6 and develop - to be carried forward into Juju 2.7+.

Revision history for this message
Tim Penhey (thumper) wrote : Re: [Bug 1824465] Re: New units fail provisioning; juju 2.5.x switches to the Standard SKU (Azure provider)

yep.

On 25/06/19 2:11 PM, Joel Sing wrote:
> Thanks for the fix - I presume this will also be addressed in the 2.6.x
> series?
>

Revision history for this message
Anastasia (anastasia-macmood) wrote :

Merge PR, bringing this change into 2.6: https://github.com/juju/juju/pull/10375

Revision history for this message
Anastasia (anastasia-macmood) wrote :

Merge PR, bringing this change into develop: https://github.com/juju/juju/pull/10376/

Changed in juju:
status: Fix Committed → Fix Released
Changed in juju:
status: Fix Released → Fix Committed
milestone: 2.5.8 → 2.7-beta1
Revision history for this message
Haw Loeung (hloeung) wrote :

So with the latest changes:

| juju model-config load-balancer-sku-name=Basic

Changed in juju:
status: Fix Committed → Fix Released
Revision history for this message
Haw Loeung (hloeung) wrote :

Unfortunately, even with load-balancer-sku-name=Basic, it's failing now with this:

"message": "{\r\n \"error\": {\r\n \"code\": \"InvalidRequestContent\",\r\n \"message\": \"The
request content was invalid and could not be deserialized: Required property 'name' not found in 'sku'.\"\r\n }\r\n}"

See https://private-fileshare.canonical.com/~hloeung/tmp/MQDFS8Cx.png

Revision history for this message
Haw Loeung (hloeung) wrote :

Juju 2.6.10 btw.

Ian Booth (wallyworld)
Changed in juju:
milestone: 2.7-beta1 → 2.8-rc1
assignee: Tim McNamara (tim-clicks) → nobody
status: Fix Released → Triaged
Ian Booth (wallyworld)
Changed in juju:
milestone: 2.8-rc1 → 2.8.1
Changed in juju:
status: Triaged → In Progress
assignee: nobody → Yang Kelvin Liu (kelvin.liu)
Revision history for this message
Yang Kelvin Liu (kelvin.liu) wrote :

Hi Haw,

I was trying to reproduce the SKU name missing. But it seems all working for me.

https://pastebin.ubuntu.com/p/pSxZb85fSW/

What I did was:

# change model config for the SKU after added model;
$ juju model-config load-balancer-sku-name=Basic

# then deploy;
$ juju deploy cs:~jameinel/ubuntu-lite

Would you help to give reproducible steps?

Revision history for this message
Ian Booth (wallyworld) wrote :

You need to start out with a 2.4 model and upgrade.
The initial model needs to be created with the old basic SKU type.
On the upgraded model, Juju will by default use the Standard SKU type, so you use the model config to switch it back to Basic. But in making API calls to Azure, Juju is missing places where the SKU type needs to be passed in, so there's a mismatch.

Revision history for this message
Yang Kelvin Liu (kelvin.liu) wrote :

https://github.com/juju/juju/pull/11695 will be landed to 2.7 to fix this issue.

Changed in juju:
status: In Progress → Fix Committed
Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.