Backports can't build-depend on other backports

Bug #888665 reported by Felix Geyer on 2011-11-10
92
This bug affects 14 people
Affects Status Importance Assigned to Milestone
Launchpad itself
High
Adam Conrad

Bug Description

Since NotAutomatic is enabled for backports (>= natty), backports can't build-depend on other backports anymore.

For example https://launchpad.net/ubuntu/+source/teeworlds/0.6.0-2~natty1 is in dep-wait even though bam 0.4.0-2~natty1is available from natty-backports:
After installing, the following source dependencies are still unsatisfied:
bam(inst 0.2.0-4 ! >= wanted 0.4)

This can probably be solved by using the aptitude sbuild resolver for backports.
At least that's what the man page recommends and what Debian is using for experimental and backports.

> The aptitude resolver [...] is suited to more complex situations, such as
> building packages for the experimental distribution, where packages
> need installing from multiple suites (unstable and experimental).

Adam Conrad (adconrad) wrote :

The recommendation to use the aptitude resolver would make a lot more sense if the sbuild in launchpad-buildd had such a thing. So, I guess we have three possible courses of action here:

1) update the sbuild in launchpad-buildd (planned, but not currently a priority) and use different resolvers for different pockets (seems a bit sketchy)
2) Flip automatic installation on for backports on the buildds (may cause other issues, if we end up upgrading things we don't want)
3) Hack the current sbuild in lp-buildd to recognize the "build-dep not met but available in backports" situation and DTRT.

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in launchpad:
status: New → Confirmed
Evan Broder (broder) wrote :

Current versions of sbuild (I'm looking at 0.62.5-1ubuntu2) have changed how the apt resolver works. It now creates a dummy package whose dependencies match the build-depends of the package being built, puts that dummy package into an apt repository, adds the apt-repository to the sources.list, and installs the dummy package using apt.

Using a resolver with that behavior would fix this bug, so we may not need to use a different resolver on different pockets. I think this makes Adam's (1) the right long-term solution.

But given recent discussion, I don't think people generally have well-formed expectations of how the backports pocket *should* work (see the most recent Tech Board meeting, for example). I would prefer that we override NotAutomatic on builds in the backport pocket quickly than fix the resolver slowly, since that would be consistent with past behavior on backports builds.

Micah Gersten (micahg) wrote :

I think that's fine for a temporary fix, but I think long term, this should behave more like experimental where the build dependency needs to be expressed to prevent using unintended backported packages.

Scott Kitterman (kitterman) wrote :

Actually I think Evan's description is what's wanted.

If the version from backports is needed to satisfy the dependency, then it ought to be pulled from there. I believe that's what apt will do.

Evan Broder (broder) wrote :

The "apt" resolver in the version of sbuild currently used on LP isn't that intelligent. sbuild parses out the build-depends, and runs "apt-get install pkg1 pkg2 pkg3" - you can see it in the failed build log in the bug's description. It's not intelligent enough to go to a non-default apt source.

Makes sense because the apt changes to support NotAutomatic are in Natty
and LP runs on Lucid.

Evan Broder (broder) wrote :

No, no, I don't think it's inherently an issue with the version of apt - this code is all running within the build chroot, and uses the chroot's apt. When sbuild passes off the build-depends to apt, apt has no awareness of specific version requirements, so it pulls using the default source instead of looking to sources that are pinned lower.

Once apt has finished pulling from its default sources, sbuild goes back and says "Hey, you screwed up, I don't have a new enough version of whatever" and bails.

tags: added: soyuz-build
Changed in launchpad:
status: Confirmed → Triaged
importance: Undecided → High
Iain Lane (laney) wrote :

So it sounds like upgrading sbuild in launchpad-buildd would be enough ­— i.e. the apt resolver in newer sbuilds will DTRT for all pockets. Is that correct?

Martin Pool (mbp) wrote :

upgraded to which version?

we recently took a patch from slangasek to support muiltarch; we could
fairly easily deploy another

Iain Lane (laney) wrote :

Ah, that I haven't analysed. What is the relationship between sbuild in launchpad-buildd and upstream sbuild? It seems like it is not just a snapshot (or at least it is quite an old snapshot; upstream looks pretty different now) so a wholesale update would be quite difficult.

The change needed is to make the resolver more smart, as described in comment #3. Some relevant bits upstream are

  http://anonscm.debian.org/gitweb/?p=buildd-tools/sbuild.git;a=blob;f=lib/Sbuild/ResolverBase.pm;h=4ed99f5f21d0fc1241932184e51d6ed2ed3cdd33;hb=HEAD#l503

  http://anonscm.debian.org/gitweb/?p=buildd-tools/sbuild.git;a=blob;f=lib/Sbuild/AptResolver.pm;h=6356394dc787149d0136bdb54e050c63c784e565;hb=HEAD

William Grant (wgrant) wrote :

It's a hacked up version of DSA sbuild as used on Debian's buildds in 2004.

I ported everything to a largely-unmodified distro version in <https://code.launchpad.net/~wgrant/launchpad/use-system-sbuild> and <https://code.launchpad.net/~wgrant/ubuntu/lucid/sbuild/extended-result>. But it relies on LP DDEB support, which is close but not there yet.

Francis J. Lacoste (flacoste) wrote :

Escalated by Bryce:

 This is a significant limitation on what can be provided
to users via backports. Solving this directly improves the Ubuntu
product for end users who need to install newer software on the stable
Ubuntu release.

tags: added: escalated
Changed in launchpad:
importance: High → Critical
Adam Conrad (adconrad) wrote :

I should probably take this bug, as I keep having discussions about it out-of-band and, I suspect, I'll ultimately be deeply involved with making it happen anyway.

Changed in launchpad:
assignee: nobody → Adam Conrad (adconrad)
Iain Lane (laney) wrote :

I'm now convinced that the correct solution is for the buildds to /ignore/ NotAutomatic and always use whatever's available.

Adam Conrad (adconrad) wrote :

For backports, I agree entirely that ignoring NotAutomatic is just lovely, and likely the right thing. I understand that this is also being discussed as a solution to turning on NotAutomatic for proposed, however.

The above solution is perfectly reasonable if we also bring britney into play for proposed->updates migration in stable series as we plan to do for proposed->release in the development series. Without that, though, we end up building against bits that we might then be kicking out of proposed. Of course, this happens today, and doesn't seem to be much of a big deal, we manually sort it out when we mess it up, but it would still be nicer to have automated installability checks to go with this whole plan.

Scott Kitterman (kitterman) wrote :

That's probably right for the devel release, but not for backports.

Scott Kitterman (kitterman) wrote :

On Friday, July 27, 2012 07:53:49 AM Adam Conrad wrote:
> For backports, I agree entirely that ignoring NotAutomatic is just
> lovely, and likely the right thing. I understand that this is also
> being discussed as a solution to turning on NotAutomatic for proposed,
> however.

For backports, dependencies from backports should only be used for building if
the version in -release doesn't satisfy the minimum version requirements (like
Debian Experimental, AIUI).

Adam Conrad (adconrad) wrote :

The reason to keep experimental packages as isolated from each other as possible is that they might get ripped out without notice and leave other bits uninstallable. Do we really have that same policy with backports? I really see no issues with just declaring that, like any other Ubuntu pocket, backports is internally consistent, and let it depend on itself across the board.

(I'm partially biased here, in that implementing the other bit is significantly more work, and not on the near-term TODO, but I honestly don't see the argument for keeping backports compartmentalised, since we don't deliver them in a compartmentalised fashion, but as part of a widely-populated repository)

Scott Kitterman (kitterman) wrote :

I think just ignoring NotAutomatic is definitely better than the current
situation, so if the other's a lot harder and ignoring could get done soon, I
definitely don't want to block progress.

My theory is that given that support for backports is different than
release/updates/security, I think the pocket should be design to minimize the
amount of interdependency to that users only get what they explicitly asked
for plus the least additional depends from backports that can be reasonably
managed.

That's consistent with why we made backports NotAutomatic to begin with. I
don't think users should enable backports and then eat the entire pocket.
They should get the new stuff (plus updates to that stuff) they explicitly ask
for and ~nothing else.

I'm not implementing it though, so I will repeat for emphasis that ignoring is
WAY better than the current situation and I don't want perfect to block
better.

Francis J. Lacoste (flacoste) wrote :

Bryce actually de-escalated this bug in favour of bug 192076 on 2012-01-10

tags: removed: escalated
William Grant (wgrant) on 2012-12-20
Changed in launchpad:
importance: Critical → High
James Page (james-page) wrote :

I'd really like to start backporting Jenkins to older releases as it moves so quickly upstream; however as Jenkins is actually more than a single package this problem blocks me right now as I need to build-dep on updated dependencies of the core package.

Martin Packman (gz) wrote :

In order to backport juju-core we'll need go 1.1, which is not a good candidate for SRUing back to precise. If we could backport the newer go, and build from that, backporting juju-core to precise would be possible.

Antonio Rosales (arosales) wrote :

+1 to Martin Packman's comment 23. Being able to have juju core in 12.04 backports would immensely help the juju-core in LTS release story.

-Thanks,
Antonio

dino99 (9d9) wrote :

An other example of impossible backport: https://bugs.launchpad.net/ubuntu/+source/hplip/+bug/1248188

This is a show stopper, and needs a fix asap.

Adam Conrad (adconrad) wrote :

As can be seen with the builds at https://launchpad.net/ubuntu/+source/hplip/3.13.9-1~ubuntu12.04.1 this has been fixed for the precise chroots, and I'm uploading the rest now.

Changed in launchpad:
status: Triaged → Fix Released
Adam Conrad (adconrad) wrote :

(For the record, I went with the "just ignore NotAutomatic" solution, in the form of pinning backports to 500, the same as any other pocket... We can revisit this another time if we really need/want more fine-grained control)

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Related blueprints