Fuel for OpenStack

Centos dashboard/project error 500 on infinity to integer conversion

Bug #1250227 reported by Vladimir Kuklin
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Low
Dmitry Pyzhov
Fuel for OpenStack 4.0

Bug Description

Deploy simple centos+cinder+compute

Open Horizon, go to project page

get the error:

OverflowError at /project/
cannot convert float infinity to integer
Request Method: GET
Request URL: http://172.16.1.2/dashboard/project/
Django Version: 1.4.8
Exception Type: OverflowError
Exception Value:
cannot convert float infinity to integer
Exception Location: /usr/lib/python2.6/site-packages/django/template/defaulttags.py in render, line 460
Python Executable: /usr/bin/python
Python Version: 2.6.6
Python Path:
['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..',
 '/usr/lib64/python26.zip',
 '/usr/lib64/python2.6',
 '/usr/lib64/python2.6/plat-linux2',
 '/usr/lib64/python2.6/lib-tk',
 '/usr/lib64/python2.6/lib-old',
 '/usr/lib64/python2.6/lib-dynload',
 '/usr/lib64/python2.6/site-packages',
 '/usr/lib/python2.6/site-packages',
 '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info',
 '/usr/share/openstack-dashboard/openstack_dashboard']
Server time: Пнд, 11 Ноя 2013 21:36:42 +0000

Environment:

Request Method: GET
Request URL: http://172.16.1.2/dashboard/project/

Django Version: 1.4.8
Python Version: 2.6.6
Installed Applications:
('openstack_dashboard',
 'django.contrib.contenttypes',
 'django.contrib.auth',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'compressor',
 'horizon',
 'openstack_dashboard.dashboards.project',
 'openstack_dashboard.dashboards.admin',
 'openstack_dashboard.dashboards.settings',
 'openstack_auth',
 'openstack_dashboard.dashboards.router')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'horizon.middleware.HorizonMiddleware',
 'django.middleware.doc.XViewMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')

Template error:
In template /usr/lib/python2.6/site-packages/horizon/templates/horizon/common/_limit_summary.html, error at line 27
   cannot convert float infinity to integer
   17 : </div>

   18 :

   19 : <div class="d3_quota_bar">

   20 : <div class="d3_pie_chart" data-used="{% widthratio usage.limits.totalRAMUsed usage.limits.maxTotalRAMSize 100 %}"></div>

   21 : <strong>{% trans "RAM" %} <br />

   22 : {% blocktrans with used=usage.limits.totalRAMUsed|mb_float_format available=usage.limits.maxTotalRAMSize|mb_float_format %}Used <span> {{ used }} </span> of <span> {{ available }} </span>{% endblocktrans %}

   23 : </strong>

   24 : </div>

   25 :

   26 : <div class="d3_quota_bar">

   27 : <div class="d3_pie_chart" data-used=" {% widthratio usage.limits.totalFloatingIpsUsed usage.limits.maxTotalFloatingIps 100 %} "></div>

   28 : <strong>{% trans "Floating IPs" %} <br />

   29 : {% blocktrans with used=usage.limits.totalFloatingIpsUsed|intcomma available=usage.limits.maxTotalFloatingIps|intcomma %}Used <span> {{ used }} </span> of <span> {{ available }} </span>{% endblocktrans %}

   30 : </strong>

   31 : </div>

   32 :

   33 : <div class="d3_quota_bar">

   34 : <div class="d3_pie_chart" data-used="{% widthratio usage.limits.totalSecurityGroupsUsed usage.limits.maxSecurityGroups 100 %}"></div>

   35 : <strong>{% trans "Security Groups" %} <br />

   36 : {% blocktrans with used=usage.limits.totalSecurityGroupsUsed|intcomma available=usage.limits.maxSecurityGroups|intcomma%}Used <span> {{ used }} </span> of <span> {{ available }} </span>{% endblocktrans %}

   37 : </strong>

Traceback:
File "/usr/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  136. response = response.render()
File "/usr/lib/python2.6/site-packages/django/template/response.py" in render
  104. self._set_content(self.rendered_content)
File "/usr/lib/python2.6/site-packages/django/template/response.py" in rendered_content
  81. content = template.render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in render
  140. return self._render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in _render
  134. return self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in render
  823. bit = self.render_node(node, context)
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  74. return node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  123. return compiled_parent._render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in _render
  134. return self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in render
  823. bit = self.render_node(node, context)
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  74. return node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  62. result = block.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in render
  823. bit = self.render_node(node, context)
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  74. return node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  62. result = block.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in render
  823. bit = self.render_node(node, context)
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  74. return node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  155. return self.render_template(self.template, context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render_template
  137. output = template.render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in render
  140. return self._render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in _render
  134. return self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/base.py" in render
  823. bit = self.render_node(node, context)
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  74. return node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/defaulttags.py" in render
  460. return str(int(round(ratio)))

Exception Type: OverflowError at /project/
Exception Value: cannot convert float infinity to integer

Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

{u'release': u'4.0', u'nailgun_sha': u'cacc3c1db665db4c9b02b19479678dba2299a71d', u'ostf_sha': u'0c72c99fadf6408312bea0385e5a2042b0fcc751', u'astute_sha': u'df6ddea3abc93fbe1cab9b4534d4d5e9508c95d6', u'fuellib_sha': u'f77672d70516795995522d5689ec67546615c4a2'}

Vladimir Kuklin (vkuklin)
Changed in fuel:
status: Triaged → Fix Committed
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

verify on 4-0-48

Revision history for this message
Anastasia Palkina (apalkina) wrote :

Reproduced on ISO #124
"release": "4.0",
"nailgun_sha": "8d80f823c38c2af6dc98173bcbe348d022960a3d",
"ostf_sha": "cf48dac2a6e7ad284fc93c529f3d1e4668504028",
"astute_sha": "ae026938f272f69afbe89c9900bf1c3df483557c",
"fuellib_sha": "687a554eb9b6ae4dcc114f34e9690e601b40610c"

Environment: CentOS, Ceph for images. 1 controller, 1 compute and 3 ceph nodes

Request Method: GET
Request URL: http://172.16.0.6/dashboard/project/
Django Version: 1.4.8
Exception Type: OverflowError
Exception Value:

cannot convert float infinity to integer

Exception Location: /usr/lib/python2.6/site-packages/django/template/defaulttags.py in render, line 460
Python Executable: /usr/bin/python
Python Version: 2.6.6
Python Path:

['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..',
 '/usr/lib64/python26.zip',
 '/usr/lib64/python2.6',
 '/usr/lib64/python2.6/plat-linux2',
 '/usr/lib64/python2.6/lib-tk',
 '/usr/lib64/python2.6/lib-old',
 '/usr/lib64/python2.6/lib-dynload',
 '/usr/lib64/python2.6/site-packages',
 '/usr/lib/python2.6/site-packages',
 '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info',
 '/usr/share/openstack-dashboard/openstack_dashboard']

Server time: Wed, 11 Dec 2013 09:53:45 +0000

In template /usr/lib/python2.6/site-packages/horizon/templates/horizon/common/_limit_summary.html, error at line 27

Changed in fuel:
status: Fix Committed → Confirmed
assignee: nobody → Vladimir Kuklin (vkuklin)
Vladimir Kuklin (vkuklin)
summary: - Centos dashboard/project error 500
+ Centos dashboard/project error 500 on infinity to float conversion
summary: - Centos dashboard/project error 500 on infinity to float conversion
+ Centos dashboard/project error 500 on infinity to integer conversion
Changed in fuel:
status: Confirmed → Triaged
importance: Medium → High
Revision history for this message
Anastasia Palkina (apalkina) wrote :
Download full text (5.9 KiB)

Environment:

Request Method: GET
Request URL: http://172.16.0.6/dashboard/project/

Django Version: 1.4.8
Python Version: 2.6.6
Installed Applications:
('openstack_dashboard',
 'django.contrib.contenttypes',
 'django.contrib.auth',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'compressor',
 'horizon',
 'openstack_dashboard.dashboards.project',
 'openstack_dashboard.dashboards.admin',
 'openstack_dashboard.dashboards.settings',
 'openstack_auth',
 'openstack_dashboard.dashboards.router')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'horizon.middleware.HorizonMiddleware',
 'django.middleware.doc.XViewMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')

Template error:
In template /usr/lib/python2.6/site-packages/horizon/templates/horizon/common/_limit_summary.html, error at line 27
   cannot convert float infinity to integer
   17 : </div>

   18 :

   19 : <div class="d3_quota_bar">

   20 : <div class="d3_pie_chart" data-used="{% widthratio usage.limits.totalRAMUsed usage.limits.maxTotalRAMSize 100 %}"></div>

   21 : <strong>{% trans "RAM" %} <br />

   22 : {% blocktrans with used=usage.limits.totalRAMUsed|mb_float_format available=usage.limits.maxTotalRAMSize|mb_float_format %}Used <span> {{ used }} </span> of <span> {{ available }} </span>{% endblocktrans %}

   23 : </strong>

   24 : </div>

   25 :

   26 : <div class="d3_quota_bar">

   27 : <div class="d3_pie_chart" data-used=" {% widthratio usage.limits.totalFloatingIpsUsed usage.limits.maxTotalFloatingIps 100 %} "></div>

   28 : <strong>{% trans "Floating IPs" %} <br />

   29 : {% blocktrans with used=usage.limits.totalFloatingIpsUsed|intcomma available=usage.limits.maxTotalFloatingIps|intcomma %}Used <span> {{ used }} </span> of <span> {{ available }} </span>{% endblocktrans %}

   30 : </strong>

   31 : </div>

   32 :

   33 : <div class="d3_quota_bar">

   34 : <div class="d3_pie_chart" data-used="{% widthratio usage.limits.totalSecurityGroupsUsed usage.limits.maxSecurityGroups 100 %}"></div>

   35 : <strong>{% trans "Security Groups" %} <br />

   36 : {% blocktrans with used=usage.limits.totalSecurityGroupsUsed|intcomma available=usage.limits.maxSecurityGroups|intcomma%}Used <span> {{ used }} </span> of <span> {{ available }} </span>{% endblocktrans %}

   37 : </strong>

Traceback:
File "/usr/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  136. response = response.render()
File "/usr/lib/python2.6/site-packages/django/template/response.py" in render
  104. self._set_content(self.rendered_content)
File "/usr/lib/python2.6/site-packages/django/template/response.py" in rendered_content
  81. content = template.render(context)
Fil...

Read more...

Revision history for this message
Mike Scherbakov (mihgen) wrote :

Folks, pls use paste.openstack.org for such a long log pastes

Vladimir Kuklin (vkuklin)
Changed in fuel:
assignee: Vladimir Kuklin (vkuklin) → Dmitry Pyzhov (lux-place)
Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

For some reason OpenStack thinks that project has 'Inf' assigned floating IP addresses. I've added one floating to project, issues dissapears. I've removed this floating, issue did not come back. Looks like root cause is somewhere in project creation phase.

Dmitry Pyzhov (dpyzhov)
Changed in fuel:
assignee: Dmitry Pyzhov (lux-place) → Fuel Library Team (fuel-library)
Revision history for this message
Andrey Korolyov (xdeller) wrote :

not able to reproduce w/ fresh cluster

Bogdan Dobrelya (bogdando)
Changed in fuel:
status: Triaged → Invalid
Vladimir Kuklin (vkuklin)
Changed in fuel:
status: Invalid → Triaged
Revision history for this message
Vladimir Sharshov (vsharshov) wrote :

Able to reproduce 2 days ago, but yesterday in same case all was fine (#159).

Revision history for this message
Mike Scherbakov (mihgen) wrote :

Can it be caching issue? I was unable to reproduce this.

Revision history for this message
Vladimir Sharshov (vsharshov) wrote :

I think problem with OSTF.

Step to reproduce:

- Deploy simple centos: 1 node controller + 2 nodes with role compute
- Run OSTF tests.
- Open Horizon, go to project page and get 500 error. Check dashboard-django.request.log

If do not run OSTF tests before open project page in Horizon — all is open without any errors.

Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

167 - execute many times on Centos and neutron (vlan/gre) in simple mode (1-1-1) - can not reproduce

Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

OSTf was executed near 6 times (all tests - sanity - smoke-platform)

Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

AAAAAAA!

mysql> select resource,in_use from quota_usages;
+-----------------+--------+
| resource | in_use |
 +-----------------+--------+
| security_groups | 0 |
| instances | 0 |
| ram | 0 |
| cores | 0 |
| fixed_ips | 0 |
| floating_ips | -1 |
 +-----------------+--------+
6 rows in set (0.00 sec)

Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

Reproduced on simple centos deployment with default settings (nova-network/flatdhcp). Issue appeared after ostf tests run.

Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

Okey. OSTF tries to remove two floating ips. For some reason it sends two requests for each ip: https://bugs.launchpad.net/fuel/+bug/1262795

Nova (or just nova-network) seems to have epic bug with counters. It decreases in_use.floating_ips counter by two in this case. This leads to -1 allocated ip address in database.

Horizon has another bug related to counters. It tries to draw nice-looking diagram of allocated floating ips. And it dies with 500 error when tries to draw -1 allocated ip.

I'm not sure if this issue exists on quantum. I'm not sure if we should report a bug to nova team. I'm not sure if we should report a bug to horizon team.

Changed in fuel:
status: Triaged → In Progress
assignee: Fuel Library Team (fuel-library) → Dmitry Pyzhov (lux-place)
Revision history for this message
Mike Scherbakov (mihgen) wrote :

Great troubleshooting. Please consider a bit more research on the topic, and file bugs to Nova and other OpenStack projects.

Revision history for this message
Sergey Vasilenko (xenolog) wrote :

upper value for drawing each diagram Horizon gets from database. From Quota's table, corresponding for tenant.
If Quota value for this item is infinity, in database will be '-1' value.

Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

Sergey, we should test if infinity quota causes the same issue, but I think it is fine.

Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

Bug is fixed. Action items: create bugs in nova-network and horizon projects

Changed in fuel:
importance: High → Low
Dmitry Pyzhov (dpyzhov)
Changed in fuel:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.