Smart Package Manager

pycurl does not fail on authentication error

Bug #244453 reported by Rehan Khan
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Smart Package Manager
Fix Released
Undecided
Unassigned
Smart Package Manager 1.3.1
smart (Ubuntu)
Fix Released
Undecided
Unassigned
Lucid
Fix Released
Undecided
Unassigned
Maverick
Fix Released
Undecided
Unassigned
Natty
Fix Released
Undecided
Unassigned

Bug Description

Request for SRU
===============

0. This is fixed in oneiric with smart 1.4 (the fix was released in 1.3.1 upstream)

1. Statement explaining the impact

Smart doesn't catch 401 (and other 40x) errors when trying to download files from a repository. As a result, the html error page presented by the server is downloaded as if it were the requested file.
This usually leads to a weird error about the GPG signature being incorrect or being made with an unknown key:
"""
-> https://ahasenack:*@private-ppa.launchpad.net/landscape/lds-stable/ubuntu/dists/lucid/Release
Release
################################################################# [ 7%]
error: Channel 'lucid - main' signed with unknown key
"""
And these are the contents of the downloaded file:
"""
root@amra:/var/lib/smart/channels# cat aptsync-c57be5d14aae4fbf00383bbe1479fbb9%%https:__ahasenack:<email address hidden>net_landscape_lds-stable_ubuntu_dists_lucid_Release.gpg
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Authorization Required</title>
</head><body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
are authorized to access the document
requested. Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
<hr>
<address>Apache/2.2.14 (Ubuntu) Server at private-ppa.launchpad.net Port 443</address>
</body></html>
root@amra:/var/lib/smart/channels#
"""

2. How has the bug been addressed
A patch was taken from trunk to make pycurl fail on http errors and then catch this error:
bzr diff -r 947..948 lp:smart

3. Patch
That patch was applied to the source package and a debdiff was generated for each ubuntu release.

4. How to reproduce
- install smartpm-core
- add this line to your sources.list:
deb https://foo:<email address hidden>/landscape/lds-trunk/ubuntu lucid main
- run smart update as root
- with the broken package, you will get a gpg signature error when that repository is fetched:
"""
-> https://foo:*@private-ppa.launchpad.net/landscape/lds-trunk/ubuntu/dists/lucid/Release
Release ########################################################################################## [ 74%]
error: Channel 'lucid - main' signed with unknown key
"""
- with the fixed package, you will get a proper 401 error:
"""
# smart update
(...)
-> https://foo:*@private-ppa.launchpad.net/landscape/lds-trunk/ubuntu/dists/lucid/Release
Release [ 66%]
error: Download of Release failed for channel 'lucid - main': The requested URL returned error: 401
"""

5. Regression potential

- In both cases, before and after the patch, smart update would fail. With the patch, it fails with the correct error message.

- Without the patch, if the repository's signature isn't checked, the error is even more cryptic:
"""
warning: Component 'main' is not in Release file for channel 'lucid - main'
"""
That's because the release file that was downloaded is actually the 401 html error page.

- the patch has a test

- I don't know of any other side effects of setting handle.setopt(pycurl.FAILONERROR, 1), which is the main fix here.

Original bug description follows
================================

Imported: http://tracker.labix.org/issue310

Reason for Import: Patch Review

further details: https://blueprints.launchpad.net/smart/+spec/bug-reporting-migration

msg1153 (view) Author: peter-endian Date: 2007-06-21.13:35:42

When you use python-curl, curl downloads the error message if authentication
fails and stores it as the respective xml file, with which smart is not very
happy of course.

The attached patch (against 0.50) sets the curl configuration option which
causes curl to fail if an authentication error occurrs, which then causes smart
to print out the error and consider the download not to be ok instead of writing
the errormessage down to the file.

See original description
Revision history for this message
Rehan Khan (rasker) wrote :
Anders F Björklund (afb)
Changed in smart:
milestone: none → 1.3.1
Anders F Björklund (afb)
Changed in smart:
status: New → Fix Committed
Anders F Björklund (afb)
Changed in smart:
status: Fix Committed → Fix Released
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Fix for lucid.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Correct attachment now (s/lucid/lucid-proposed/ in the changelog entry).

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Now for maverick.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

natty.

Andreas Hasenack (ahasenack)
description: updated
Andreas Hasenack (ahasenack)
Changed in smart (Ubuntu):
status: New → Fix Released
description: updated
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Fixed natty debdiff.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :
Revision history for this message
Andreas Hasenack (ahasenack) wrote :
Revision history for this message
Clint Byrum (clint-fewbar) wrote : Please test proposed package

Hello Rehan, or anyone else affected,

Accepted smart into natty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in smart (Ubuntu Natty):
status: New → Fix Committed
tags: added: verification-needed
Changed in smart (Ubuntu Maverick):
status: New → Fix Committed
Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Hello Rehan, or anyone else affected,

Accepted smart into maverick-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in smart (Ubuntu Lucid):
status: New → Fix Committed
Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Hello Rehan, or anyone else affected,

Accepted smart into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Revision history for this message
Andreas Hasenack (ahasenack) wrote :
Download full text (3.3 KiB)

Verified for lucid:
root@ls1-lucid:~# smart update aptsync-901cc275ff5db7780238d2aa60f94a6c
Loading cache...
Updating cache... ########################################################################################## [100%]

Fetching information for 'lucid - main'...
-> https://ahasenackk:*@private-ppa.launchpad.net/landscape/lds-trunk/ubuntu/dists/lucid/Release.gpg
Release.gpg ########################################################################################## [ 33%]
-> https://ahasenackk:*@private-ppa.launchpad.net/landscape/lds-trunk/ubuntu/dists/lucid/Release
Release ########################################################################################## [ 66%]
error: Channel 'lucid - main' signed with unknown key
(...)
(update package)

root@ls1-lucid:~# smart update aptsync-901cc275ff5db7780238d2aa60f94a6c
Loading cache...
Updating cache... ########################################################################################## [100%]

Fetching information for 'lucid - main'...
-> https://ahasenackk:*@private-ppa.launchpad.net/landscape/lds-trunk/ubuntu/dists/lucid/Release.gpg
Release.gpg [ 0%]
-> https://ahasenackk:*@private-ppa.launchpad.net/landscape/lds-trunk/ubuntu/dists/lucid/Release
Release [ 0%]
error: Download of Release failed for channel 'lucid - main': The requested URL returned error: 401

root@ls1-lucid:~# dpkg -l python-smartpm
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==================================-==================================-====================================================================================
ii python-smartpm 1.2-5ubuntu0.2 Python library of the Smart Package Manager

root@ls1-lucid:~# apt-cache policy python-smartpm
python-smartpm:
  Installed: 1.2-5ubuntu0.2
  Candidate: 1.2-5ubuntu0.2
  Version table:
 *** 1.2-5ubuntu0.2 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid-proposed/main Packages
        100 ...

Read more...

Revision history for this message
Andreas Hasenack (ahasenack) wrote :
Download full text (3.4 KiB)

Verified in maverick using the package in proposed.

Reproducing the error (note: the authenticated repo I'm using is lucid, but it doesn't matter because all I want is the 401 error):
root@ls3:~# /usr/share/smart/smart update aptsync-934815f120a8ad05a851477558b537ba
Loading cache...
Updating cache... ########################################################################################## [100%]

Fetching information for 'lucid - main'...
-> https://ahasenack:*@private-ppa.launchpad.net/landscape/lds-proposed/ubuntu/dists/lucid/Release.gpg
Release.gpg ########################################################################################## [ 33%]
-> https://ahasenack:*@private-ppa.launchpad.net/landscape/lds-proposed/ubuntu/dists/lucid/Release
Release ########################################################################################## [ 66%]
error: Channel 'lucid - main' signed with unknown key

Updating the package:

root@ls3:~# apt-get install python-smartpm
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
  python-smartpm
1 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.
Need to get 258kB of archives.
After this operation, 0B of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu/ maverick-proposed/main python-smartpm i386 1.3-1ubuntu0.2 [258kB]
Fetched 258kB in 0s (390kB/s)
(Reading database ... 56327 files and directories currently installed.)
Preparing to replace python-smartpm 1.3-1ubuntu0.1 (using .../python-smartpm_1.3-1ubuntu0.2_i386.deb) ...
Unpacking replacement python-smartpm ...
Processing triggers for python-support ...
Setting up python-smartpm (1.3-1ubuntu0.2) ...
Processing triggers for python-support ...

Running again, now we get a proper 401:

root@ls3:~# /usr/share/smart/smart update aptsync-934815f120a8ad05a851477558b537ba
Loading cache...
Updating cache... ########################################################################################## [100%]

Fetching information for 'lucid - main'...
-> https://ahasenack:*@private-ppa.launchpad.net/landscape/lds-proposed/ubuntu/dists/lucid/Release.gpg
Release.gpg [ 0%]
-> https://ahasenack:*@private-ppa.launchpad.net/landscape/lds-proposed/ubuntu/dists/lucid/Release
Release ...

Read more...

Revision history for this message
Andreas Hasenack (ahasenack) wrote :
Download full text (3.6 KiB)

Verified with the proposed package for natty.

Reproducing the problem (using a lucid test repository to trigger the 401, it doesn't matter that it's not natty):
root@li37-87:~# smart update aptsync-4256fab382bd1817c0c8c568a38ae5ea
Loading cache...
Updating cache... ########################################################################################## [100%]

Fetching information for 'lucid - main'...
-> https://ahasenack:*@private-ppa.launchpad.net/landscape/landscape-testing/ubuntu/dists/lucid/Release.gpg
Release.gpg ########################################################################################## [ 33%]
-> https://ahasenack:*@private-ppa.launchpad.net/landscape/landscape-testing/ubuntu/dists/lucid/Release
Release ########################################################################################## [ 66%]
error: Channel 'lucid - main' signed with unknown key

Updating the package:

root@li37-87:~# apt-get install smartpm-core python-smartpm
Reading package lists... Done
Building dependency tree
Reading state information... Done
smartpm-core is already the newest version.
The following packages were automatically installed and are no longer required:
  python-psycopg2 python-egenix-mxdatetime python-egenix-mxtools libpq5
Use 'apt-get autoremove' to remove them.
The following packages will be upgraded:
  python-smartpm
1 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
Need to get 285 kB of archives.
After this operation, 0 B of additional disk space will be used.
Do you want to continue [Y/n]?
Get:1 http://security.ubuntu.com/ubuntu/ natty-proposed/main python-smartpm i386 1.3-1.3ubuntu0.2 [285 kB]
Fetched 285 kB in 0s (384 kB/s)
(Reading database ... 28103 files and directories currently installed.)
Preparing to replace python-smartpm 1.3-1.3ubuntu0.1 (using .../python-smartpm_1.3-1.3ubuntu0.2_i386.deb) ...
Unpacking replacement python-smartpm ...
Processing triggers for python-support ...
Setting up python-smartpm (1.3-1.3ubuntu0.2) ...
Processing triggers for python-support ...

Trying smart again, now we get a nice 401 error:
root@li37-87:~# smart update aptsync-4256fab382bd1817c0c8c568a38ae5ea
Loading cache...
Updating cache... ########################################################################################## [100%]

Fetching information for 'lucid - main'...
-> https://ahasenack:*@private-ppa.launchpad.net/landscape/landscape-testing/ubuntu/dists/lucid/Release.gpg
Release.gpg ...

Read more...

Martin Pitt (pitti)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package smart - 1.2-5ubuntu0.2

---------------
smart (1.2-5ubuntu0.2) lucid-proposed; urgency=low

  * Handle authentication errors when using pycurl, giving a meaningful
    error message. (LP: #244453)
 -- Andreas Hasenack <email address hidden> Wed, 07 Sep 2011 11:12:42 -0300

Changed in smart (Ubuntu Lucid):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package smart - 1.3-1ubuntu0.2

---------------
smart (1.3-1ubuntu0.2) maverick-proposed; urgency=low

  * Handle authentication errors when using pycurl, giving a meaningful
    error message. (LP: #244453)
 -- Andreas Hasenack <email address hidden> Sat, 10 Sep 2011 18:37:03 -0400

Changed in smart (Ubuntu Maverick):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package smart - 1.3-1.3ubuntu0.2

---------------
smart (1.3-1.3ubuntu0.2) natty-proposed; urgency=low

  * Handle authentication errors when using pycurl, giving a meaningful
    error message. (LP: #244453)
 -- Andreas Hasenack <email address hidden> Sat, 10 Sep 2011 19:44:13 -0300

Changed in smart (Ubuntu Natty):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.