lp:ubuntu/cloud-init is not buildable by bzr-builder

[Neil Wilson]

Not sure where the fault lies with this one, however a simple recipe file based on the package branch:

 bzr-builder format 0.3 deb-version {debupstream}-0ubuntu0+{revno}
lp:ubuntu/cloud-init

Will not rebuild the package correctly.

Yet pulling down the source package normally and building by hand works perfectly.

I get the same problem whether on launchpad or using bzr-builder locally.

https://code.launchpad.net/~brightbox/+archive/experimental/+buildjob/2485999

[Aaron Bentley (abentley)]

It looks as though the branch contains the changes that are provided by the quit patches. Is this the case?

[James Westby (james-w)]

On Wed, 20 Apr 2011 14:21:40 -0000, Aaron Bentley <email address hidden> wrote:
> It looks as though the branch contains the changes that are provided by
> the quit patches. Is this the case?

Yes.

Thanks,

James

[James Westby (james-w)]

On Wed, 20 Apr 2011 14:21:40 -0000, Aaron Bentley <email address hidden> wrote:
> It looks as though the branch contains the changes that are provided by
> the quit patches. Is this the case?

There is however no .pc directory, so quilt doesn't know that.

This looks like a case of broken content to me as things currently
stand.

Thanks,

James

[Deryck Hodge (deryck)]

I'm not clear if this could be a bug in launchpad or not? Can we close the launchpad task as invalid? Or is Launchpad doing something wrong here to mess up the branch content?

[Aaron Bentley (abentley)]

This is a problem with the content of the branch, as created by a person, not anything Launchpad did.

[Clint Byrum (clint-fewbar)]

Are you certain this is a problem with the branch?

The branch matches what is in the Ubuntu archives exactly. bzr bd -S from the branch and apt-get source produce identical .dsc/.debian.tar.gz/.orig.tar.gz files.

It seems more appropriate then that bzr-builder should be able to read this the same way bzr-buildpackage does .. not that this branch should somehow be changed to suit bzr-builder. I notice that all of the patches are to catch the branch up with upstream trunk. Maybe whats really broken is that you need to have your recipe remove the patches from the build entirely.

[Clint Byrum (clint-fewbar)]

Neil, I'm marking this bug as Incomplete. Please try removing the patches in the build recipe, as the build log you pasted seems to suggest that they are simply conflicting with the branch content, which is a normal condition. You can either add 'run rm debian/patches/*', or create a new branch of the cloud-init packaging and merge that in before you merge in lp:cloud-init.

[Neil Wilson (neil-aldur)]

That's hardly user friendly.

Surely if the builder is to be of any use whatsoever it should build the Ubuntu source branch with a simple command.

Having to manipulate a source package upload to get it to build is totally counter-intuitive.

[James Westby (james-w)]

Hi Jelmer,

Perhaps the applying of quilt patches in bzr-builder can be more graceful in its
handling of this state.

That is:

  * patches in debian/patches
  * Those patches already applied
  * No .pc directory

Thanks,

James

[Clint Byrum (clint-fewbar)]

Excerpts from Neil Wilson's message of Mon May 16 20:26:10 UTC 2011:
> That's hardly user friendly.
>
> Surely if the builder is to be of any use whatsoever it should build the
> Ubuntu source branch with a simple command.
>
> Having to manipulate a source package upload to get it to build is
> totally counter-intuitive.

There's not much we can do if the source package patches conflicts
directly with upstream. The build log is quite clear which patches won't
apply, so they can be selectively removed if some need to stay. Ultimately
you have 4 different versions of the code:

1 upstream .orig
2 upstream .orig + packaging patches
3 upstream NEW + packaging patches
4 upstream NEW

Picking which one to build could be simpler, thats true. But the problem
is that there's some conflicting, duplicated delta between 2 and 3 that
must be hand merged because the patches are not applied, so the common
version is unknown.

I stand by my original assessment, that while its not easy, its necessary
to be able to be clear about which patches you want to apply.

[James Westby (james-w)]

On Mon, 16 May 2011 22:18:37 -0000, Clint Byrum <email address hidden> wrote:
> There's not much we can do if the source package patches conflicts
> directly with upstream. The build log is quite clear which patches won't
> apply, so they can be selectively removed if some need to stay. Ultimately
> you have 4 different versions of the code:
>
> 1 upstream .orig
> 2 upstream .orig + packaging patches
> 3 upstream NEW + packaging patches
> 4 upstream NEW
>
> Picking which one to build could be simpler, thats true. But the problem
> is that there's some conflicting, duplicated delta between 2 and 3 that
> must be hand merged because the patches are not applied, so the common
> version is unknown.
>
> I stand by my original assessment, that while its not easy, its necessary
> to be able to be clear about which patches you want to apply.

Is there some confusion here? This case isn't about updating the package
to a new upstream release. This is just about rebuilding the current
Ubuntu package, so what's in debian/patches should apply, otherwise the
packaging is broken.

What is causing the issue is that debuild and bzr-builder apply the
quilt patches in slightly different ways, with bzr-builder failing if
the patches are already applied and there is no .pc directory, and
debuild no failing in that case.

It's my opinion that when using bzr + dpkg v3 (quilt), the bzr tree
should have patches applied and a .pc directory, as that allows you to
directly work with quilt when getting the branch. However, given that
debuild accepts the current branch as input, bzr-builder probably should
too, as it isn't really doing anything different.

This is yet another case where the mismatch between quilt patches and
bzr bites us, so I'd like for it to go away by natively supporting
changes against a base in bzr.

Thanks,

James

[Neil Wilson (neil-aldur)]

"This is just about rebuilding the current Ubuntu package,"

Exactly.

Nobody is expecting an auto merge here.

This is the standard distribution package as uploaded automatically to Bzr being built on that distribution by the Launchpad builder system.

And currently for this package it doesn't work.

[Clint Byrum (clint-fewbar)]

Excerpts from James Westby's message of Tue May 17 02:08:47 UTC 2011:
> On Mon, 16 May 2011 22:18:37 -0000, Clint Byrum <email address hidden> wrote:
> > There's not much we can do if the source package patches conflicts
> > directly with upstream. The build log is quite clear which patches won't
> > apply, so they can be selectively removed if some need to stay. Ultimately
> > you have 4 different versions of the code:
> >
> > 1 upstream .orig
> > 2 upstream .orig + packaging patches
> > 3 upstream NEW + packaging patches
> > 4 upstream NEW
> >
> > Picking which one to build could be simpler, thats true. But the problem
> > is that there's some conflicting, duplicated delta between 2 and 3 that
> > must be hand merged because the patches are not applied, so the common
> > version is unknown.
> >
> > I stand by my original assessment, that while its not easy, its necessary
> > to be able to be clear about which patches you want to apply.
>
> Is there some confusion here? This case isn't about updating the package
> to a new upstream release. This is just about rebuilding the current
> Ubuntu package, so what's in debian/patches should apply, otherwise the
> packaging is broken.
>
> What is causing the issue is that debuild and bzr-builder apply the
> quilt patches in slightly different ways, with bzr-builder failing if
> the patches are already applied and there is no .pc directory, and
> debuild no failing in that case.
>
> It's my opinion that when using bzr + dpkg v3 (quilt), the bzr tree
> should have patches applied and a .pc directory, as that allows you to
> directly work with quilt when getting the branch. However, given that
> debuild accepts the current branch as input, bzr-builder probably should
> too, as it isn't really doing anything different.
>
> This is yet another case where the mismatch between quilt patches and
> bzr bites us, so I'd like for it to go away by natively supporting
> changes against a base in bzr.

I see, apologies to those I misunderstood.

So yes, I think bzr-builder should employ the same standards as debuild,
since it is the de-facto way packages are built, while builder is trying
to emulate it.

It would also make sense that bzr-buildpackage would warn the user that
their package has changes applied, but no .pc directory. Even better
would be if it could actually *create* that .pc directory, as I am
somewhat confused as to how to do that simply.

[Scott Moser (smoser)]

On Tue, 17 May 2011, James Westby wrote:

> What is causing the issue is that debuild and bzr-builder apply the
> quilt patches in slightly different ways, with bzr-builder failing if
> the patches are already applied and there is no .pc directory, and
> debuild no failing in that case.

I'd much prefer to have no .pc directory in packaging directories. I
realize its probably just personal preference, but it seems silly at best
to revision control a .pc directory. At worst, it is actually problematic
to me, making it *much* more difficult to easily see the changes from one
revision of a package to another.

I personally think having a way to easily create the .pc directory from
the current (clean) state makes more sense.

See the debian/rules target of 'quilt-setup' for cloud-init that does
that.

[Scott Moser (smoser)]

I'm moving this to confirmed in bzr-builder.

That said, I would be willing to have the lp:ubuntu/cloud-init branch fixed because I'd like for it to build.
I personally really hate .pc directories being versioned, but would live with it or consider moving off quilt-3.0.

somewhat related, is that cloud-init is on the failed importer list [http://package-import.ubuntu.com/status/].
I *think* that if we just wiped all ubuntu/cloud-init history, and re-imported from scratch, both that issue and this issue would be resolved.

[Scott Moser (smoser)]

generally, i think that bzr-builder should be better, and allow for no .pc directory but applied patches.

However, cloud-init's ubuntu trunk is now an upstream snapshot, so:
$ cat > cloud-init.recipe <<"EOF"
# bzr-builder format 0.4 deb-version {debupstream}-0ubuntu0+{revno}
lp:ubuntu/cloud-init
EOF
$ bzr dailydeb cloud-init.recipe working-dir

That "worked for me" today on precise, with, building '0.6.3~bzr502-0ubuntu1'
bzr-builder 0.7.2-0ubuntu1

using 'format 0.3' works also (so probably older bzr-builders would work).