Quantal failed to install: ubiquity crashed in apt/progress/text.py in pulse: OverflowError: Python int too large to convert to C long

Bug #1030278 reported by Jung-Kyu Park on 2012-07-28
36
This bug affects 3 people
Affects Status Importance Assigned to Milestone
the python apt bindings
New
Undecided
Unassigned
python-apt (Ubuntu)
High
James Hunt
Quantal
High
James Hunt

Bug Description

install problem of ubuntu12.10 quantal quetzal

ProblemType: Bug
DistroRelease: Ubuntu 12.10
Package: ubiquity 2.11.18
ProcVersionSignature: Ubuntu 3.5.0-6.6-generic 3.5.0
Uname: Linux 3.5.0-6-generic i686
ApportVersion: 2.4-0ubuntu5
Architecture: i386
CasperVersion: 1.320
Date: Sat Jul 28 20:45:15 2012
InstallCmdLine: noprompt cdrom-detect/try-usb=true persistent file=/cdrom/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.lz quiet splash -- maybe-ubiquity
LiveMediaBuild: Ubuntu 12.10 "Quantal Quetzal" - Alpha i386 (20120724.2)
ProcEnviron:
 LANGUAGE=ko_KR.UTF-8
 TERM=linux
 PATH=(custom, no user)
 LANG=ko_KR.UTF-8
SourcePackage: ubiquity
UpgradeStatus: No upgrade log present (probably fresh install)

Jung-Kyu Park (bagjunggyu) wrote :
Sasa Paporovic (melchiaros) wrote :

In UbiquitySyslog.txt:

Jul 28 11:40:09 ubuntu ubiquity: Error in function pulse
Jul 28 11:40:09 ubuntu plugininstall.py: Exception during installation:
Jul 28 11:40:09 ubuntu plugininstall.py: Traceback (most recent call last):
Jul 28 11:40:09 ubuntu plugininstall.py: File "/usr/lib/ubiquity/ubiquity/install_misc.py", line 276, in pulse
Jul 28 11:40:09 ubuntu plugininstall.py: AcquireProgress.pulse(self, owner)
Jul 28 11:40:09 ubuntu plugininstall.py: File "/usr/lib/python3/dist-packages/apt/progress/text.py", line 167, in pulse
Jul 28 11:40:09 ubuntu plugininstall.py: for worker in owner.workers:
Jul 28 11:40:09 ubuntu plugininstall.py: OverflowError: Python int too large to convert to C long
Jul 28 11:40:09 ubuntu plugininstall.py:
Jul 28 11:45:15 ubuntu ubiquity: Gtk-Message: Not loading module "atk-bridge": The functionality is provided by GTK natively. Please try to not load it.

summary: - install problem of ubuntu12.10 quantal quetzal
+ install problem of ubuntu12.10 quantal quetzal - Error in function pulse
summary: - install problem of ubuntu12.10 quantal quetzal - Error in function pulse
+ Quantal failed to install: ubiquity crashed in apt/progress/text.py in
+ pulse: OverflowError: Python int too large to convert to C long
affects: ubiquity (Ubuntu) → apt (Ubuntu)
affects: apt (Ubuntu) → python-apt (Ubuntu)
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in python-apt (Ubuntu):
status: New → Confirmed
tags: added: rls-q-incoming
Changed in python-apt (Ubuntu):
importance: Undecided → High
Colin Watson (cjwatson) on 2012-09-19
tags: removed: rls-q-incoming
Steve Langasek (vorlon) on 2012-09-26
Changed in python-apt (Ubuntu Quantal):
assignee: nobody → James Hunt (jamesodhunt)
Colin Watson (cjwatson) wrote :

This is a regression from Python 2. Here's a minimal test case:

<cjwatson@sarantium ~>$ python -c 'import apt_pkg; print(apt_pkg.size_to_str(2147483648))'
2147 M
<cjwatson@sarantium ~>$ python3 -c 'import apt_pkg; print(apt_pkg.size_to_str(2147483648))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: Python int too large to convert to C long

Changed in python-apt (Ubuntu Quantal):
status: Confirmed → Triaged
Colin Watson (cjwatson) on 2012-09-30
Changed in python-apt (Ubuntu Quantal):
milestone: none → ubuntu-12.10
Barry Warsaw (barry) wrote :

Hmm:

% python -c 'import apt_pkg; print(apt_pkg.size_to_str(2147483648))'
2147 M
% python3 -c 'import apt_pkg; print(apt_pkg.size_to_str(2147483648))'
2147 M

But even if that was the problem, then it confirms my suspicion as discussed with James on IRC, that some exception state is not being checked. If you look at the traceback in UbiquitySyslog.txt, the error is getting raised during iteration over owner.workers. Which makes no sense <wink>. Most likely this OverflowError is getting set some place but not checked at the point of error, and the traceback is a red herring. That would mean there are two bugs here (although as you can see, I cannot reproduce Colin's test case).

Barry Warsaw (barry) wrote :

$ python3 -c 'import apt_pkg; print(apt_pkg.size_to_str(2147483648000))'
2147 G
$ python3 -c 'import apt_pkg; print(apt_pkg.size_to_str(2147483648000000))'
2147 T
$ python3 -c 'import apt_pkg; print(apt_pkg.size_to_str(2147483648000000000))'
2147 P
$ python3 -c 'import apt_pkg; print(apt_pkg.size_to_str(2147483648000000000000))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: Python int too large to convert to C long

Jean-Baptiste Lallement (jibel) wrote :

Barry, did you try to reproduce on 32bit or 64bit ? This bug and all the duplicates are on 32bit.

On Oct 01, 2012, at 08:08 PM, Jean-Baptiste Lallement wrote:

>Barry, did you try to reproduce on 32bit or 64bit ? This bug and all the
>duplicates are on 32bit.

D'oh. Nope, only tried on 64bit. Let me try again.

Barry Warsaw (barry) wrote :

Okay, this is clearly a problem in StrSizeToStr() and I think both bugs are related to this. First, StrSizeToStr() never checks whether an exception occurs during conversion, so that's likely the problem of the bogus traceback. Second, StrSizeToStr() needs to PyLong_Check() *before* it PyInt_Check()s because in Python 3, the latter is aliased to the former, however, if the object is an *actual* PyLong, it will be convertable to a double (as per PyLong_AsDouble()) for more values than it will be convertable to a long (as per PyInt_AsLong(), a.k.a. in Python 3, PyLong_asLong()).

The attached diff should fix both problems. I've tested it with various values in an i386 chroot and it seems to work correctly now.

Just don't try to convert 21474836480000000000000000000. I guess SizeToStr() can't handle anything over thousands of yottabytes. Darn, I might have to return my new hard drive. This patch doesn't do much about that.

Barry Warsaw (barry) wrote :
tags: added: patch
Barry Warsaw (barry) wrote :

Here's a test case.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-apt - 0.8.7ubuntu2

---------------
python-apt (0.8.7ubuntu2) quantal; urgency=low

  * python/string.cc, tests/test_lp1030278.py: Fix StrSizeToStr() so that
    1) it first checks for PyLong-ness so that in Python 3 on i386, it
    will be able to convert larger numbers (via doubles rather than ints);
    2) before doing the conversions through the apt API, check to see if a
    Python exception occurred, e.g. OverflowError, and return an error
    condition in that case instead of masking it. (LP: #1030278)
 -- Barry Warsaw <email address hidden> Mon, 01 Oct 2012 18:04:35 -0400

Changed in python-apt (Ubuntu Quantal):
status: Triaged → Fix Released
James Hunt (jamesodhunt) wrote :

Hi Barry,

Thanks for this - we seem to have collided slightly on this one! :)

Now that we've got a fix in quantal, I've undone my changes, but have raised an MP for additional checks and a formal test for size_to_str(). See:

lp:~jamesodhunt/python-apt/test-for-size_to_str

You're right about the yottabytes issue - see bug 1059783 raised yesterday.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers