unclear error message for quota in horizon (possibly taking wrong one from logs)

[Dafna Ron]

I tried running 100 large instances.
horizon shows the flowing error:

Error: Quota exceeded for cores,instances,ram: Requested 800, but already used 0 of 20 cores (HTTP 413) (Request-ID: req-0c854c89-89d9-48ce-96e2-eca07bbbc8f5)

the first part of the message is fine (Quota exceeded for cores,instances,ram) but the second part is a bit unclear...
I asked to run 100 instances with the largest flavor (my host cannot support that), so the 800 value is not very clear
also, 'but already used 0 of 20 cores' means none of the cores were used.

this is the complete log, and I think that we possibly take the wrong errors:

[root@opens-vdsb ~(keystone_admin)]# egrep 0c854c89-89d9-48ce-96e2-eca07bbbc8f5 /var/log/*/*
/var/log/horizon/horizon.log:Recoverable error: Quota exceeded for cores,instances,ram: Requested 800, but already used 0 of 20 cores (HTTP 413) (Request-ID: req-0c854c89-89d9-48ce-96e2-eca07bbbc8f5)
/var/log/nova/api.log:2013-09-10 16:35:09.049 WARNING nova.compute.api [req-0c854c89-89d9-48ce-96e2-eca07bbbc8f5 5f38d619dfe744f0a8e08818033fc37e 89f7caf549e04aec85c3a8737a43a37c] cores,instances,ram quota exceeded for 89f7caf549e04aec85c3a8737a43a37c, tried to run 100 instances. Can only run 2 more instances of this type.
/var/log/nova/api.log:2013-09-10 16:35:09.049 INFO nova.api.openstack.wsgi [req-0c854c89-89d9-48ce-96e2-eca07bbbc8f5 5f38d619dfe744f0a8e08818033fc37e 89f7caf549e04aec85c3a8737a43a37c] HTTP exception thrown: Quota exceeded for cores,instances,ram: Requested 800, but already used 0 of 20 cores
/var/log/nova/api.log:2013-09-10 16:35:09.050 INFO nova.osapi_compute.wsgi.server [req-0c854c89-89d9-48ce-96e2-eca07bbbc8f5 5f38d619dfe744f0a8e08818033fc37e 89f7caf549e04aec85c3a8737a43a37c] 10.35.101.10 "POST /v2/89f7caf549e04aec85c3a8737a43a37c/servers HTTP/1.1" status: 413 len: 373 time: 0.1629190

i think that we should be taking 'tried to run 100 instances. Can only run 2 more instances of this type.'

[Victoria Martinez de la Cruz (vkmc)]

In default OpenStack deployments the largest flavor is xlarge. Are you trying to launch 100 large or xlarge instances?

If it's the second one, then the '800' value comes from the fact that xlarge flavor consist of 8VCPUs. Then, if you are using this flavor for the 100 instances, you are trying to allocate 800 VCPUs.

I guess that it calculates how many VCPUs you can use (20 in this case) and if the amount you are asking for exceeds the max, it just notify you about it.

Try launching a few (let's say... 2) small instances first, just to use some VCPUs, and then launch the 100 xlarge instances again. The error message now should say something like "Error: Quota exceeded for cores,instances,ram: Requested 800, but already used 2 of 20 cores". Does that happen?

Probably we could get a better message though.

[Dafna Ron (dron-3)]

the bug is for the error only :)

[Gabriel Hurley (gabriel-hurley)]

Can definitely have a better message.

[Tihomir Trifonov (ttrifonov)]

The error message is defined in Nova:

nova/nova/exception.py:

1077 class TooManyInstances(QuotaError):
1078: msg_fmt = _("Quota exceeded for %(overs)s: Requested %(req)s,"
1079 " but already used %(used)d of %(allowed)d %(resource)s")

and it depends on parameters. Also, if there are more than 1 exceeded quota, Nova checks for the first one and returns the "used X of allowed Y" only for the first item, instead adding all them to the error message:

            resource = overs[0]
            used = quotas[resource] - headroom[resource]
            total_allowed = used + headroom[resource]
            overs = ','.join(overs)
            ......
            raise exception.TooManyInstances(overs=overs,
                                             req=requested[resource],
                                             used=used, allowed=total_allowed,
                                             resource=resource)

Here used and total_allowed is for the overs[0] resource only. Thus the message show multiple quotas exceeded, but only 1 detail.

I think there is nothing to do in Horizon for this error message.

[Sean Dague (sdague)]

Looking at the current Nova code, I believe this was addressed some time ago. Please reopen if you still see that.

[Liyingjun (liyingjun)]

There is no such issue for upstream horizon. You cannot even submit the launch instance form if the requested resources exceed the quotas using upstream horizon..