'build' should work (or support working) in a git worktree

[Scott Moser]

git build (or git build-source) may end up complaining that you have locally modified files such as:

  dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/open-iscsi_2.0.874-2ubuntu3

i'd like to build what is committed without worrying about a dirty state.
One way to do this would be worktree.

basically just:
   dir=$(mktemp -d)
   git worktree add --force $dir/worktree .
   ( cd "$dir/worktree" && debuild )
   ...
   git worktree prune

[Nish Aravamudan (nacc)]

We don't currently support building outside of dpkg-buildpackage itself. If
dpkg-buildpackage complains about untracked modifications to the source,
that is a fatal error, switching the working tree won't help afaict?

If you can provide a few more details that will help with reproducing the
feature request (which will probably be post-1.0). What does `git status`
say? Can you provide the log file from that message and the complete
message?

On Aug 28, 2017 11:31, "Scott Moser" <email address hidden> wrote:

> Public bug reported:
>
> git build (or git build-source) may end up complaining that you have
> locally modified files such as:
>
> dpkg-source: error: aborting due to unexpected upstream changes, see
> /tmp/open-iscsi_2.0.874-2ubuntu3
>
>
> i'd like to build what is committed without worrying about a dirty state.
> One way to do this would be worktree.
>
> basically just:
> dir=$(mktemp -d)
> git worktree add --force $dir/worktree .
> ( cd "$dir/worktree" && debuild )
> ...
> git worktree prune
>
> ** Affects: usd-importer
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are a member of Ubuntu
> Server Dev import team, which is subscribed to usd-importer.
> https://bugs.launchpad.net/bugs/1713549
>
> Title:
> 'build' should work (or support working) in a git work-tree
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/usd-importer/+bug/1713549/+subscriptions
>

[Robie Basak (racb)]

I think he's saying that he'd like HEAD built regardless of the state of the working tree.

I agree this should be possible but I think the user should explicitly have to force it or similar.

[Nish Aravamudan (nacc)]

Ok, that would be a dupe of bug 1706979 and the command would look
like `git ubuntu build/build-source HEAD`. I don't think it's a
'regardless' statement, but an explicit request to build a specific
treeish.

Scott, can you confirm?

[Scott Moser (smoser)]

I wasnt explicitly asking for build/build-source to take a treeish, but I suspect you'd end up scratching my itch by fixing that bug.

HEAD would be what i'd expect the default treeish to be.

I want this to work:

  echo "DIRTY FILE" > dirty.txt
  git checkout ubuntu/devel
  git build

As it is right now the build will fail because my current working directory is dirty (unexpected upstream changes).

Since i'm using git, i'd actually expect for it to build the state of git, not the state of my working directory.

The way I suggest that you do that, without "git clean -cf" was to use git worktree, and then clean up your worktree.

[Robie Basak (racb)]

On Tue, Aug 29, 2017 at 08:00:03PM -0000, Scott Moser wrote:
> Since i'm using git, i'd actually expect for it to build the state of
> git, not the state of my working directory.

I think I disagree. Like "make" and "dpkg-buildpackage", I expect "git
ubuntu build" to use the current state of my working tree by default.
Though now that we've stuck "git " in front of our wrapper, I can
understand how you might disagree.

Perhaps we should require a clean tree by default then, but allow a
--force like gbp?

[Nish Aravamudan (nacc)]

And note with pristine-tar the way it is in the branches to land today, that will happen earlier (failing with a dirty tree), because gbp is quite unhappy. We could loosen that, after landing those commits, with the knowledge that gbp will fail (which probably means we need to handle that failure more cleanly) to extract the tarballs.

I think the discussion here is exactly the question :) are we building working-tree or HEAD?

If the former, and there are changes to the upstream, there's nothing we can do, afaict. You need to convert it to a quilt patch in order for dpkg-buildpackage to succeed (that was the message from the original bug report). Changes to debian/ should still work (I believe this is actually the case before pristine-tar lands).

[Nish Aravamudan (nacc)]

I am hesitant, as well, to overload --force. At least, I see --force could mean two rather orthogonal things:

a) overwrite existing tarballs in the parent directory
b) build even though dirty

Really, though, I think the way we'd implement a) above is not with --force, but with --mechanism=<specified>? Or perhaps specified as pristine-tar,launchpad ? That will have it skip the parent dir, but use the cache. The interactions there are a bit much to ask the user to do.

[Scott Moser (smoser)]

While I realize that building HEAD is less obvious I think it is the right thing to do.
I think it builds the right habits and then ultimately makes you *less* likely to lose things, because you're committing them to git. Even if a user resorts to something like:
  git commit -a -m "" && git ubuntu build-package

I often end up with files that are in my tree from general development, things like 'out.diff' from 'git diff > out.diff' and such. building HEAD allows you to not care.

I guess another option would be:
  git build-package tree
  git build-package HEAD
  git build-package commitish

the special 'tree', which could even be the default would build working-tree.
All others would/could use worktree to create a temporary and clean directory and build there.

[Christian Ehrhardt  (paelzer)]

OTOH I like on dpkd-buildpackage that I can on a git tree do like:

start loop
  modify
  buildpackage
return loop
commit changes

And I'd even think that this is what people expect as this is what they had if e.g. doing:
$ pull-lp-source
$ git init
[...]
$ dpkg-buildpackage

I'd like an option to say "ignore all the sh§$%& I have and build head" but I'd honestly not expect it to be the default.

[Scott Moser (smoser)]

Christian, that is what i was saying with the special 'tree' argument (which could be the default).
I do think its more likely what someone wants, but i think that after having buildpackage cry that they have files in their directory i think they'd end up wanting the default as HEAD.

[Nish Aravamudan (nacc)]

So, for what it's worth, with the edge snap, I believe Scott's request is satisified.

As mentioned a few times on IRC (and possibly in this bug) -- we don't support building from a dirty tree. We could, but it's not trivial to do (with LXD), as well, git can't archive what it doesn't know about!

So for now we do build HEAD and build it in either a tempdir or a LXD container. Please do try the edge snap and confirm.

[Nish Aravamudan (nacc)]

I would request that if there is an urgent need to build a dirty tree, that someone file a new bug for that.

[Nish Aravamudan (nacc)]

NM, already filed at https://bugs.launchpad.net/usd-importer/+bug/1714560

[Robie Basak (racb)]

On Wed, Sep 20, 2017 at 11:37:00PM -0000, Nish Aravamudan wrote:
> As mentioned a few times on IRC (and possibly in this bug) -- we don't
> support building from a dirty tree. We could, but it's not trivial to do
> (with LXD), as well, git can't archive what it doesn't know about!

I'd like to support this, even if we decide it needs to be explicit with
an option. We'll need to create a temporary different git index, ask
that to be brought up to the current state of the working tree, ask for
a tree object to be created from that temporary index, and then hand the
ref of that tree to the existing build code. That is probably a separate
bug though.