[sru] lp-propose fails with a 404 error

[Guilherme Salgado]

All of a sudden lp-propose decided it was not authorized to pose as myself on Launchpad so it asked for my authorization yet again. After I granted full rights to bzr, it failed with the following error:
(extra debug log thanks to httplib2.debuglevel=1)

Notice the "send: 'GET /beta/1.0/" line; according to Leonard, that's what's causing the problem here.

I'm running Bazaar 2.2.3, fwiw.

send: 'POST /+access-token HTTP/1.1\r\nHost: launchpad.net\r\nAccept-Encoding: identity\r\nContent-Length: 188\r\nreferer: https://launchpad.net/\r\nuser-agent: Python-httplib2/$Rev$\r\n\r\n'
send: 'oauth_signature=<redacted>&oauth_consumer_key=bzr&oauth_token=<redacted>&oauth_signature_method=PLAINTEXT'
reply: 'HTTP/1.1 200 Ok\r\n'
header: Date: Mon, 24 Jan 2011 18:36:21 GMT
header: Server: zope.server.http (HTTP)
header: X-Powered-By: Zope (www.zope.org), Python (www.python.org)
header: X-Content-Type-Warning: guessed from content
header: Content-Type: text/plain;charset=utf-8
header: Content-Length: 148
header: Vary: Cookie,Authorization,Accept-Encoding
header: Via: 1.1 launchpad.net
send: 'GET /beta/1.0/ HTTP/1.1\r\nHost: api.launchpad.net\r\nAuthorization: OAuth realm="OAuth", oauth_nonce="<redacted>", oauth_timestamp="1295894181", oauth_consumer_key="bzr", oauth_signature_method="PLAINTEXT", oauth_version="1.0", oauth_token="<redacted>", oauth_signature="<redacted>"\r\naccept-encoding: gzip, deflate\r\naccept: application/vnd.sun.wadl+xml\r\nuser-agent: lazr.restfulclient 0.9.20; oauth_consumer="bzr"\r\n\r\n'
reply: 'HTTP/1.1 404 Not Found\r\n'
header: Date: Mon, 24 Jan 2011 18:36:22 GMT
header: Server: zope.server.http (HTTP)
header: X-Powered-By: Zope (www.zope.org), Python (www.python.org)
header: X-Lazr-Oopsid: OOPS-1850G1508
header: Content-Type: text/plain
header: Content-Length: 2344
header: Via: 1.1 wildcard.launchpad.net
bzr: failed to report crash using apport:
     OSError(13, 'Permission denied')
bzr: ERROR: lazr.restfulclient.errors.HTTPError: HTTP Error 404: Not Found
Response headers:
---
content-length: 2344
content-type: text/plain
date: Mon, 24 Jan 2011 18:36:22 GMT
server: zope.server.http (HTTP)
status: 404
via: 1.1 wildcard.launchpad.net
x-lazr-oopsid: OOPS-1850G1508
x-powered-by: Zope (www.zope.org), Python (www.python.org)
---
Response body:
---
Object: <canonical.launchpad.systemhomes.WebServiceApplication object at 0x8a4b6d0>, name: u'1.0'

Traceback (most recent call last):
  File "/srv/launchpad.net/production/launchpad-rev-12248/eggs/zope.publisher-3.12.0-py2.6.egg/zope/publisher/publish.py", line 131, in publish
    obj = request.traverse(obj)
  File "/srv/launchpad.net/production/launchpad-rev-12248/eggs/lazr.restful-0.15.0-py2.6.egg/lazr/restful/publisher.py", line 236, in traverse
    result = super(WebServiceRequestTraversal, self).traverse(ob)
  File "/srv/launchpad.net/production/launchpad-rev-12248/eggs/zope.publisher-3.12.0-py2.6.egg/zope/publisher/browser.py", line 542, in traverse
    ob = super(BrowserRequest, self).traverse(obj)
  File "/srv/launchpad.net/production/launchpad-rev-12248/eggs/zope.publisher-3.12.0-py2.6.egg/zope/publisher/http.py", line 455, in traverse
    ob = super(HTTPRequest, self).traverse(obj)
  File "/srv/launchpad.net/production/launchpad-rev-12248/eggs/zope.publisher-3.12.0-py2.6.egg/zope/publisher/base.py", line 261, in traverse
    obj = publication.traverseName(self, obj, entry_name)
  File "/srv/launchpad.net/production/launchpad-rev-12248/eggs/lazr.restful-0.15.0-py2.6.egg/lazr/restful/publisher.py", line 98, in traverseName
    request, ob, name)
  File "/srv/launchpad.net/production/launchpad-rev-12248/eggs/zope.app.publication-3.9.0-py2.6.egg/zope/app/publication/zopepublication.py", line 197, in traverseName
    ob2 = adapter.publishTraverse(request, nm)
  File "/srv/launchpad.net/production/launchpad-rev-12248/lib/canonical/launchpad/browser/launchpad.py", line 722, in publishTraverse
    return Navigation.publishTraverse(self, request, name)
  File "/srv/launchpad.net/production/launchpad-rev-12248/lib/canonical/launchpad/webapp/publisher.py", line 661, in publishTraverse
    nextobj = self._publishTraverse(request, name)
  File "/srv/launchpad.net/production/launchpad-rev-12248/lib/canonical/launchpad/webapp/publisher.py", line 785, in _publishTraverse
    return self._handle_next_object(nextobj, request, name)
  File "/srv/launchpad.net/production/launchpad-rev-12248/lib/canonical/launchpad/webapp/publisher.py", line 691, in _handle_next_object
    raise NotFound(self.context, name)
NotFound: Object: <canonical.launchpad.systemhomes.WebServiceApplication object at 0x8a4b6d0>, name: u'1.0'

[Martin Pool (mbp)]

The "all of a sudden" makes me suspect this was broken by changes to
the handling of 'edge' on Launchpad.

[Robert Collins (lifeless)]

On Tue, Jan 25, 2011 at 10:09 AM, Martin Pool <email address hidden> wrote:
> The "all of a sudden" makes me suspect this was broken by changes to
> the handling of 'edge' on Launchpad.

There aren't any changes - it was all rolled back.

-Rob

[Guilherme Salgado (salgado)]

I guess I'm the only one seeing this, or else we'd have seen other reports? Is there anything I can do to help figure out what is going on?

[Martin Pool (mbp)]

It would be very useful to get a traceback from bzr showing the stack
where it decided to make that request.

> bzr: failed to report crash using apport:
> OSError(13, 'Permission denied')

This might mean /var/crash has the wrong mode. (It should be 1777.)
If you can get a crash file, that will help.

The /beta/1.0 seems like it might originate from some hand-url-hacking
in the client, perhaps originating in something being hardcoded to
beta.

Do other launchpad api commands like 'register-branch' work?

--
Martin

[Martin Pool (mbp)]

bzr 2.2 is hardcoded to use the beta api (which perhaps should
change); I can't see any way it would directly ask for /1.0 under
that.

[Robert Collins (lifeless)]

Yeah, the beta api was the first one, its deprecated and first on the
block to be deleted.

[Martin Pool (mbp)]

On 27 January 2011 14:10, Robert Collins <email address hidden> wrote:
> Yeah, the beta api was the first one, its deprecated and first on the
> block to be deleted.

https://bugs.launchpad.net/bzr/+bug/708441 to remove it. Please let
us know if/when it is actually on the block.
--
Martin

[Guilherme Salgado (salgado)]

On Wed, 2011-01-26 at 23:47 +0000, Martin Pool wrote:
> It would be very useful to get a traceback from bzr showing the stack
> where it decided to make that request.
>
> > bzr: failed to report crash using apport:
> > OSError(13, 'Permission denied')
>
> This might mean /var/crash has the wrong mode. (It should be 1777.)
> If you can get a crash file, that will help.

I don't have a /var/crash file, so apport fails when attempting to
create that file (as my user doesn't have permission to do so). Was
there anything supposed to create that file with the correct mode so
that my user has the rights to write to it?

>
> The /beta/1.0 seems like it might originate from some hand-url-hacking
> in the client, perhaps originating in something being hardcoded to
> beta.
>
> Do other launchpad api commands like 'register-branch' work?

That asks me for my Launchpad password and then fails with

bzr: ERROR: Invalid url supplied to transport: "https://salgado%40canonical.com:[e-mail]/bazaar/": nonnumeric port

[Guilherme Salgado (salgado)]

The crash file I got after manually creating /var/crash

[Max Bowsher (maxb)]

I see what's breaking here. In 1.5.5, launchpadlib redefined the meaning of a service_root URI from including the trailing version component to excluding that. Yuck, that's a nasty compatibility break. At this point, the service root constants in bzr still use the old launchpadlib <= 1.5.4 form with a trailing /beta/. *However*, bzr 2.3 doesn't use a hardcoded string for the production service root any more, but uses the imported LPNET_SERVICE_ROOT, which means if you're running bzr >= 2.3, this bug is masked.

So, I think I've figured out how to fix this. We do all of our service roots via the *_SERVICE_ROOT constants imported from launchpadlib. For any service roots not present as constants, we construct them by changing the hostname of the STAGING_SERVICE_ROOT value, but we retain any path component present there. That way we always match the expectations of the launchpadlib we are actually running against.

[Martin Pool (mbp)]

On 29 January 2011 11:28, Max Bowsher <email address hidden> wrote:
> I see what's breaking here. In 1.5.5, launchpadlib redefined the meaning
> of a service_root URI from including the trailing version component to
> excluding that. Yuck, that's a nasty compatibility break. At this point,
> the service root constants in bzr still use the old launchpadlib <=
> 1.5.4 form with a trailing /beta/. *However*, bzr 2.3 doesn't use a
> hardcoded string for the production service root any more, but uses the
> imported LPNET_SERVICE_ROOT, which means if you're running bzr >= 2.3,
> this bug is masked.

It sounds like we should file a bug against launchpadlib about that?

> So, I think I've figured out how to fix this. We do all of our service
> roots via the *_SERVICE_ROOT constants imported from launchpadlib. For
> any service roots not present as constants, we construct them by
> changing the hostname of the STAGING_SERVICE_ROOT value, but we retain
> any path component present there. That way we always match the
> expectations of the launchpadlib we are actually running against.

I suppose to support our existing behavior we have to do that. Going
forward, perhaps we should just remove this. Using anything but the
default service root is a developer-oriented feature, and it may
simpler for them just to take the lp ws url from an environment
variable.

--
Martin

[Max Bowsher (maxb)]

Under normal circumstances, yes, we should file a bug. However, as the breakage occurred ages ago, between the karmic and lucid versions of launchpadlib, I'm not sure it's still worth pursuing.

[Martin Pool (mbp)]

On 30 January 2011 00:20, Max Bowsher <email address hidden> wrote:
> Under normal circumstances, yes, we should file a bug. However, as the
> breakage occurred ages ago, between the karmic and lucid versions of
> launchpadlib, I'm not sure it's still worth pursuing.

Fair enough.

--
Martin

[Martin Pool (mbp)]

We should make this the headline SRU bug for 2.2.4 into maverick

[Vincent Ladeuil (vila)]

Dear ~ubuntu-sru, we'd like to get this into maverick-updates.
Per <https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions>, we have an exception to take all of 2.2.4, conditional on running the tests during the package build and on running the tests in 2.2.4.

This is now part of our standard process so no failures are expected there.

[Vincent Ladeuil (vila)]

Jelmer kindly provided lp:~jelmer/ubuntu/maverick/bzr/sru-2.2.4

[Martin Pool (mbp)]

See <http://wiki.bazaar.canonical.com/UbuntuStableReleaseUpdates> for an overview of the process.

I believe I need to make the upload in my name so I can sign it.

I have built the source package and uploaded it to <https://launchpad.net/~bzr/+archive/sru-staging> for an extra check before putting it into maverick-proposed.

[Martin Pool (mbp)]

Due to bug 151528 I can't test the exact same package in a ppa. This is now uploaded to maverick-proposed.

[Clint Byrum (clint-fewbar)]

Hi Jelmer, Martin.

I've been reviewing the proposed diff here:

http://launchpadlibrarian.net/68515880/bzr_2.2.1-0ubuntu1_2.2.4-0ubuntu1.diff.gz

I don't see where the test suite is enabled, and on building in a clean maverick chroot, I saw no test suite run.

As I understand it, the test suite must be run on build before bzr's micro-release exception can be accepted.

[Martin Pool (mbp)]

On 10 April 2011 05:03, Clint Byrum <email address hidden> wrote:
> Hi Jelmer, Martin.
>
> I've been reviewing the proposed diff here:
>
> http://launchpadlibrarian.net/68515880/bzr_2.2.1-0ubuntu1_2.2.4-0ubuntu1.diff.gz
>
> I don't see where the test suite is enabled, and on building in a clean
> maverick chroot, I saw no test suite run.
>
> As I understand it, the test suite must be run on build before bzr's
> micro-release exception can be accepted.

Thanks for reviewing this, Clint.

Our micro-release exception says that the test suite only needs to be
enabled during build from Natty onwards[1][2]. We would like to
enable it for Maverick but the test suite depends on things in
universe there. What we should do is run the test suite from the
built package.

[1] https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions
[2] https://lists.ubuntu.com/archives/technical-board/2010-December/000627.html

Martin

[Colin Watson (cjwatson)]

Yes, we did want to run the test suite everywhere automatically but accepted the current situation as an unfortunate fact.

Can somebody please go ahead and upload this, and we can review it in the queue?

[Colin Watson (cjwatson)]

Oh, it's already in the queue; I misread. Looking.

[Martin Pool (mbp)]

I did already upload it; sorry the bug log was unclear. It is in <https://launchpad.net/ubuntu/maverick/+queue?queue_state=1&queue_text=bzr> and I think that means it now needs someone to approve it.

[Colin Watson (cjwatson)]

Accepted bzr into maverick-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Martin Pool (mbp)]

One point here: the maverick-proposed package, despite claiming to fix <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=568937>, does not seem to have a recommendation for python-launchpadlib, so 'bzr lp-propose' does not work out in a clean minimal maverick-updates install. However: the same issue seems to be present in all maverick bzrs; many machines will get python-launchpadlib through other means; and it is fixed in natty. So I think it's irrelevant to the SRU process.

[Martin Pool (mbp)]

Verification done:

I enabled maverick-proposed in a chroot, and installed bzr_2.2.4-0ubuntu1_amd64.deb.

I made a change in a branch and ran 'bzr lp-propose' which did the oauth dance with Launchpad, and then successfully created a proposal. I also deleted the credentials from under ~/.bazaar/launchpad/ and that worked ok too.

[Robert Collins (lifeless)]

I've been using 2.2.4 from proposed, and bzr-search has its test suite fail:

======================================================================
ERROR: bzrlib.plugins.search.tests.test_index.TestGraphIndexSuggestions.test_key_length_2_no_hits(BTree)
----------------------------------------------------------------------
_StringException: Text attachment: log
------------
5.902 opening working tree '/tmp/testbzr-z0SX5I.tmp'
------------
Text attachment: traceback
------------
Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/testtools/runtest.py", line 169, in _run_user
    return fn(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/testtools/testcase.py", line 507, in _run_test_method
    return self._get_test_method()()
  File "/home/robertc/.bazaar/plugins/search/tests/test_index.py", line 703, in test_key_length_2_no_hits
    list(query_index.iter_entries_starts_with(('pref', 'pref'))))
  File "/home/robertc/.bazaar/plugins/search/index.py", line 1450, in iter_entries_starts_with
    items = sorted(node.items())
AttributeError: '_LeafNode' object has no attribute 'items'
------------

I don't know if this is a regression in the SRU or not.

[Launchpad Janitor (janitor)]

This bug was fixed in the package bzr - 2.2.4-0ubuntu1

---------------
bzr (2.2.4-0ubuntu1) maverick-proposed; urgency=low

  [ Jelmer Vernooij ]
  * Update watch file to use 2.2 series.
  * New upstream release.
   + Fixes closing of leaked sockets to SSH subprocesses, which causes
     dput sftp uploads to hang. LP: #659590
   + Fixes the use of 'lp:' urls behind a http proxy. LP: #558343
   + Correctly sets the Content-Type header when http POSTing to comply
     with stricter web frameworks. LP: #665100
   + Fixes propagating tags to the master branch in a bound branch or
     heavyweight checkout. LP: #603395
   + Fixes the use of 'bzr resolve --take-other' if the file is
     involved in an unresolved text conflict. LP: #646961
   + Fixes https access with newer versions of python2.7. LP: #693880
   + Fixes crash during pack caused by a concurrent repository pack
     operation. LP: #701940
   + Fixes communication with the Launchpad web service when using
     launchpadlib >= 1.5.5. LP: #707075
   + Switches away from deprecated 'edge.launchpad.net' LP: #583667
   + Fixes resolving of content (and path) conflicts for files in subdirs.
     LP: #660935
   + Fixes nasty recursion loop while displaying branch opening error.
     LP: #687653

  [ Martin Pool ]
  * Propose for maverick SRU.
 -- Martin Pool <email address hidden> Thu, 07 Apr 2011 15:30:17 +1000

[Martin Pool (mbp)]

@Robert the SRU has now gone out so I suggest we file a new bug for that.

Looking back to bzr 2.0 I can't see that index nodes ever provided a .items() method, and nothing in index.py has changed since bzr 2.2.0. So perhaps this failure was independently present in bzr-search?

[Max Bowsher (maxb)]

Removed verification-done tag from the maverick SRU. We are now starting the SRU process for the lucid SRU for the same fix.

The lucid SRU will be accomplished via an update of bzr in lucid from 2.1.1 (release pocket) to 2.1.4 (per MicroReleaseException).

The proposed changelog entry is:

bzr (2.1.4-0ubuntu1) lucid; urgency=low

  * Update watch file to use 2.1 series.
  * New upstream release.
   + Fix file descriptors leaks in dirstate compiled extension. LP: #583486
   + Refuse to stack on repositories in incompatible formats. LP: #562380
   + Don't delete nested trees/repos. LP: #572098
   + Fix 'bzr switch' crash when a 'ConfigurableFileMerger' is used. LP: #559436
   + Fix compatibility with older smart servers. LP: #528041
   + Fix symlinks addition. LP: #192859
   + Properly unversion children of unversioned directories. LP: #494221
   + Lock configuration files in '~/.bazaar' for updates. LP: #525571
   + Fix 'bzr commit <symlink>'. LP: #128562
   + Fix `lp:` urls when behind an http proxy. LP: #558343
   + Stop using edge.launchpad.net. LP: #583667

 -- Max Bowsher <email address hidden> Tue, 17 May 2011 09:54:17 +0100

The packaging for this SRU is in lp:~bzr/ubuntu/lucid/bzr/sru

[Max Bowsher (maxb)]

Hello ~ubuntu-sru:

bzr 2.1.4-0ubuntu1 is now ready on the unapproved queue for your review:

https://launchpad.net/ubuntu/lucid/+queue?queue_state=1&queue_text=bzr

[Max Bowsher (maxb)]

This SRU has actually been processed through bug 494221 instead.

[Frank Hoks (wolfdog-joey)]

On Thu, 19 May 2011 09:19:19 -0000, Max Bowsher wrote:

> This SRU
has actually been processed through bug 494221 instead.

[Martin Pool (mbp)]

@wolfdog, if you're sick of getting mail about this, there should be a link in the bottom of the mail to unsubscribe.

[Frank Hoks (wolfdog-joey)]

@martin

no there isn't

On Thu, 19 May 2011 12:57:37 -0000, Martin
Pool wrote:

> @wolfdog, if you're sick of getting mail about this,
there should be a
> link in the bottom of the mail to unsubscribe.

[Vincent Ladeuil (vila)]

>>>>> wolfdog writes:

    > @martin
    > no there isn't

    > On Thu, 19 May 2011 12:57:37 -0000, Martin
    > Pool wrote:

    >> @wolfdog, if you're sick of getting mail about this,
    > there should be a
    >> link in the bottom of the mail to unsubscribe.

    > --
    > You received this bug notification because you are a direct subscriber
    > of the bug.
    > https://bugs.launchpad.net/bugs/707075

    > Title:
    > [sru] lp-propose fails with a 404 error

    > To unsubscribe from this bug, go to:
    > https://bugs.launchpad.net/bzr/+bug/707075/+subscribe

Here it is. Yeah, it says 'subscribe' but it should handle unsubscribe too.

[Frank Hoks (wolfdog-joey)]

I can't find it, how and were to unsubscribe.

I can't find it on
launchpad.

Who can?

On Thu, 19 May 2011 14:21:27 -0000, Vincent
Ladeuil wrote:

>>>>>> wolfdog writes:
> > @martin > no there isn't >
On Thu, 19 May 2011 12:57:37 -0000, Martin > Pool wrote: >> @wolfdog, if
you're sick of getting mail about this, > there should be a >> link in
the bottom of the mail to unsubscribe. > -- > You received this bug
notification because you are a direct subscriber > of the bug. >
https://bugs.launchpad.net/bugs/707075 [1] > Title: > [sru] lp-propose
fails with a 404 error > To unsubscribe from this bug, go to: >
https://bugs.launchpad.net/bzr/+bug/707075/+subscribe [2] Here it is.
Yeah, it says 'subscribe' but it should handle unsubscribe too.

Links:
------
[1] https://bugs.launchpad.net/bugs/707075
[2]
https://bugs.launchpad.net/bzr/+bug/707075/+subscribe

[Alexander Belchenko (bialix)]

wolfdog пишет:
> I can't find it, how and were to unsubscribe.
>
> I can't find it on
> launchpad.
>
> Who can?

After subscribe I see red circle with minus sign after my name in the
list of subscribers, and that icon allows me to unsubscribe. Do you?

[Martin Pool (mbp)]

@wolfdog, please open a support question under
https://answers.launchpad.net/launchpad and paste in the full text of
one of the problem mails, including the headers. Then just paste the
url of that question here.

Martin

[Frank Hoks (wolfdog-joey)]

I unsubscribed.

https://bugs.launchpad.net/ubuntu/+source/debconf/+bug/349469/+subscribe
[2]

On Thu, 19 May 2011 15:49:30 -0000, Martin Pool wrote:

>
@wolfdog, please open a support question under
>
https://answers.launchpad.net/launchpad [1] and paste in the full text
of
> one of the problem mails, including the headers. Then just paste
the
> url of that question here.
>
> Martin

Links:
------
[1]
https://answers.launchpad.net/launchpad
[2]
https://bugs.launchpad.net/ubuntu/+source/debconf/+bug/349469/+subscribe