OpenStack Image Registry and Delivery Service (Glance)

glance add can fail for images larger than 2 GB on some Python installations (RHEL 6.2)

Reported by Chris Hoge on 2012-02-17
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
Critical
Eoghan Glynn

Bug Description

Some Python installations, in particular for RHEL 6.2 Python 2.6, are not 64 bit clean, and "glance add" will fail. For example, where testimage.img is larger than 2GB

glance --debug add name="testimage" disk_format=raw is_public=true < testimage.img
python: Objects/fileobject.c:2117: readahead_get_line_skip: Assertion `skip+len < 2147483647' failed.

The underlying problem is an INT_MAX macro in the fileobject.c method 'readahead_get_line_skip' uses a 32 bit value instead of a 64 bit value. The Python file iterator eventually calls down to this method, which causes a crash for large file sizes.

A workaround for RHEL users is to comment out the "elif self._iterable(body):" section for the image_iterator method in glance/common/client.py and fall back to the ImageBodyIterator.

While not directly a bug with Glance, it does impact a subset of users not running Ubuntu systems.

Chris Hoge (hoge) wrote :

Correction: we ran into this problem on RHEL 6.2.

summary: glance add can fail for images larger than 2 GB on some Python
- installations (RHEL 6.1)
+ installations (RHEL 6.2)
description: updated

Hi,
I've run into this issue on ubuntu oneiric with python-glance version:

2012.1~e3-0ubuntu0~ppa1~oneiric1

Jay Pipes (jaypipes) on 2012-02-17
Changed in glance:
status: New → Confirmed
importance: Undecided → Critical
milestone: none → essex-4
assignee: nobody → Eoghan Glynn (eglynn)
Eoghan Glynn (eglynn) on 2012-02-17
Changed in glance:
status: Confirmed → In Progress
Eoghan Glynn (eglynn) wrote :

Chris,

What exact version of the python-2.6 package are you using on RHEL?

Can you copy'n'paste the output of the following:

$ uname -a
$ rpm -qa | grep python-2.6

Thanks,
Eoghan

Eoghan Glynn (eglynn) wrote :

Linuxdatacenter,

Similar question for you, can you paste in the output of:

$ uname -a
$ dpkg -s python2.6

Cheers,
Eoghan

Chris Hoge (hoge) wrote :

Results of
$ uname -a
$ rpm -qa | grep python-2.6

Linux cn3 2.6.32-220.4.1.el6.x86_64 #1 SMP Thu Jan 19 14:50:54 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
python-2.6.6-29.el6.x86_64

Eoghan Glynn (eglynn) wrote :

Thanks for the quick response Chris.

Now I'm having difficulty re-producing the issue against glance latest, and in fact code inspection suggests that if falling through to the ImageBodyIterator clause in glance/common/client.py acts as a work-around in your case, then the problem should have been resolved by this commit:

  https://github.com/openstack/glance/commit/40b5ed73

as the codepaths followed in the presence of the read attr would be equivalent.

Is the glance version you're testing against slightly behind the curve? You can check very quickly with:

$ grep -A 1 "elif self._iterable" glance/common/client.py

Output like:

        elif self._iterable(body):
            return utils.chunkreadable(body)

indicates that you have the relevant commit, whereas:

        elif self._iterable(body):
            return body

indicates that you need to update to latest.

Can you try the above grep and get back to me?

Thanks,
Eoghan

Chris Hoge (hoge) wrote :

Yes, my branch is about 7 days old now. I'll update and test (and be sure to be on latest when I run into problems :-).

Eoghan Glynn (eglynn) wrote :

Thanks Chris - let me know who you fare after updating, so that I can close off the bug if indeed it's not reproducible against latest.

Chris Hoge (hoge) wrote :

That fixed it here. Thanks.

I also submitted a bug against the RHEL 6 build of Python, fwiw.

Eoghan Glynn (eglynn) wrote :

Thanks Chris, closing off this bug as invalid as the fix is already in.

Eoghan Glynn (eglynn) wrote :

Actually Fix Committed is a more accurate status, as the bug was orignally kosher but had been fixed as a side-effect of:

 https://github.com/openstack/glance/commit/40b5ed73

Changed in glance:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2012-02-29
Changed in glance:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-04-05
Changed in glance:
milestone: essex-4 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers