Please backport tahoe-lafs (1.8.3-0ubuntu1) from oneiric

[Daira Hopwood]

Please backport 1.8.2-3fakesync1 from Oneiric. See http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/NEWS.rst?rev=5046 for the upstream changelog between versions 1.6.1 and 1.8.2. Several packaging bugs have also been fixed since then.

[Zooko Wilcox-O'Hearn (zooko)]

Yes, please! This would be very good. Tahoe-LAFS has a strong policy of backward compatibility and excellent quality control, so I would not expect any compatibility problems or nasty surprises for Lucid users upgrading from Tahoe-LAFS 1.6.1 to 1.8.2.

[pataquets (pataquets)]

+1

[Evan Broder (broder)]

In order to ensure a safe upgrade path for users with the backport installed, our policies will require us to additionally backport tahoe-lafs to Maverick and Natty. I've gone ahead and opened bug tasks for those releases.

Since tahoe-lafs has no reverse-dependencies, we need verification that the package successfully builds, installs, and runs on the older releases. I've uploaded test builds to my PPA (https://launchpad.net/~broder/+archive/backports-tests). Please verify that the backports build successfully, install, and run on all of Lucid, Maverick, and Natty.

Once you've verified that, please change the status of the bug to Confirmed. In the mean time, I'm going to set it to Incomplete to indicate that the backporters team has done its initial once-over.

[Daira Hopwood (daira)]

I can test this for Maverick. There was a separate bug ticket for Maverick (bug 834354) which I will mark as a duplicate.

[pataquets (pataquets)]

Lucid test:
Apt-purged tahoe and dependencies OK.
Added ppa successfully and apt update'd.
When trying to install tahoe-lafs (1.8.3-0ubuntu1~lucid1~ppa1) apt-get complains:

tahoe-lafs:
 Depends: python-foolscap but will not be installed
  Depends: python-simplejson (>=2.1.1) but will install 2.0.9-1build1
 Depends: python-pycryptopp but will not be installed

Versions installed/available in my repos:
python-foolscap: 0.5.1+dfsg-0ubuntu1 (lucid universe)
python-simplejson: 2.0.9-1build1 (lucid main) Ubuntuone-client, apport and others depend on it.
python-pycryptopp: 0.5.17-1 (lucid universe) Same problem as in Natty (lp bug #811721)

I haven't seen further instructions on the PPA guide to test further.

[Zooko Wilcox-O'Hearn (zooko)]

I'm a developer of the upstream Tahoe-LAFS project. Our canonical documentation of which versions of which dependencesi we require is this file:

https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/src/allmydata/_auto_deps.py?annotate=blame

Here is the version of that file from Tahoe-LAFS v1.9.0 (released October 31, 2011):

https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/src/allmydata/_auto_deps.py?annotate=blame&rev=5357

And here is the version of that file from Tahoe-LAFS v1.8.3 (released September 13, 2011):

https://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/src/allmydata/_auto_deps.py?annotate=blame&rev=5015

(Here is the Parade of Release Notes: https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Doc#TheParadeofReleaseNotes .)

Tahoe-LAFS upstream requires simplejson >= 1.4, pycryptopp >= 0.5.20, and foolscap >= 0.6.1.

[Evan Broder (broder)]

Thanks for the feedback, everyone. Let's look at the missing dependencies one at a time.

> simplejson >= 1.4

It looks like the Debian maintainer at some point bumped the simplejson dependency from (>= 1.4) to (>= 2.1.1). Zooko (or anyone), do you have any idea why the dependency might have gotten bumped so much higher than tahoe-lafs requires? If that dependency is artificially high, then relaxing it back to (>= 1.4) would greatly simplify the backport. However, if 2.1.1 or higher is actually required, that will require a very invasive backport - simplejson has several dozen reverse dependencies that would need to be tested.

> foolscap >= 0.6.1

A new enough version of foolscap was first made available in Natty. Before we can proceed with the tahoe-lafs backport, we will need to backport foolscap. Can someone please open a separate bug for that?

If you have either a Precise machine or an ubuntu-dev-tools daily build installed (https://launchpad.net/~udt-developers/+archive/daily), I recommend using the requestbackport script, which will determine all of the testing that needs to be done for the backport.

> pycryptopp >= 0.5.20

Again, there seems to be a disagreement in the packaging. The package depends on python-pycryptopp (>= 0.5.29) - any thoughts on why the dependency is higher?

In any case, it looks like Natty picked up 0.5.29 by virtue of a security update (bug #811721). I'm going to see about doing the same for Lucid and Maverick. Fixing this through a security update or SRU would be sufficient to move the backport forward.

[Zooko Wilcox-O'Hearn (zooko)]

> It looks like the Debian maintainer at some point bumped the simplejson dependency from (>= 1.4) to (>= 2.1.1). Zooko (or anyone), do you have any idea why the dependency might have gotten bumped so much higher than tahoe-lafs requires?

Well, ideally we would ask the Debian maintainers -- Bert Agaz and Micah Anderson (per http://packages.debian.org/wheezy/tahoe-lafs ). Even *more* ideally, they would have written down in a changelog entry why they chose those values. I'll go look for such a thing in the Debian packaging history.

As an upstream maintainer, I'm pretty confident that our upstream requirements of simplejson >= 1.4, foolscap >= 0.6.1, and pycryptopp >= 0.5.20 are sufficient.

[Daira Hopwood (daira)]

This changeset released in simplejson 2.1: http://code.google.com/p/simplejson/source/diff?spec=svn212&r=212&format=side&path=/trunk/simplejson/_speedups.c , looks security-relevant.

(I have to say that the simplejson maintainers don't seem to be very good at providing meaningful stand-alone commit comments; also they mix formatting changes and security-relevant changes in the same patch, as here for example: http://code.google.com/p/simplejson/source/detail?r=122 , which to me is very bad form.)

[Daira Hopwood (daira)]

"> pycryptopp >= 0.5.20

Again, there seems to be a disagreement in the packaging. The package depends on python-pycryptopp (>= 0.5.29) - any thoughts on why the dependency is higher?"

As far as I know, the tahoe-lafs package need not depend on 0.5.29, that just happened to be the version of pycryptopp that Natty packaged in order to fix bug 811721.

[Daira Hopwood (daira)]

Maverick has the following missing dependencies:

 tahoe-lafs : Depends: python-foolscap (>= 0.6.1-3) but 0.5.1+dfsg-0ubuntu1 is to be installed
              Depends: python-pycryptopp (>= 0.5.29-1) but 0.5.17-1 is to be installed

It does have python-simplejson, version 2.1.1-1.

[Evan Broder (broder)]

I've updated bug #811721 for Lucid and Maverick, and uploaded the new pycryptopp version to both of them. They'll be subject to review by the SRU team, and then will require verification as well.

Once that process has been finished and the foolscap backport has been dealt with, I'll try relaxing the simplejson dependency for the Lucid backport of Tahoe.

[Zooko Wilcox-O'Hearn (zooko)]

Thanks for working on this, Evan!

[Daira Hopwood (daira)]

I reviewed all upstream changes in simplejson between 2.0.9 (the version in Lucid) and 2.1.1 for potentially breaking changes:

- there was a new feature, object_pairs_hook, that added a fair amount of code. This caused a regression that was caught before release: http://code.google.com/p/simplejson/source/detail?r=195
- http://code.google.com/p/simplejson/source/detail?r=181 causes a new exception JSONDecodeError to be raised on a decoding error rather than ValueError. That could in theory have a compatibility impact, but it's unlikely because JSONDecodeError inherits from ValueError. So to be broken, a client would need to be depending on the exact class ValueError being raised; 'except ValueError' would continue to work.

I didn't see any other changes that might be risky for compatibility.

[Evan Broder (broder)]

I'm closing the maverick-backports task on this bug due to Ubuntu 10.10 (Maverick Meerkat) no longer being supported.

It looks like you requested a backport to Ubuntu 10.04 (Lucid Lynx). Now that Ubuntu 10.10 is no longer supported, it may be possible to backport this package directly to Ubuntu 10.04, without requiring backports to intervening releases.

I'm going to assume you are primarily interested in the Ubuntu 10.04 backport, and mark the newer bug tasks as Incomplete. If this is incorrect, please feel free to adjust the status yourself, or comment and I will do so.

This bug is being manipulated by a bot. If you feel the change was made in error, please feel free to re-open the bug. However, backports requests for Ubuntu 10.10 (Maverick Meerkat) are no longer being accepted.